Private Clouds Better for Data Warehousing than Public Clouds

Cloud solutions are useful when additional computing power is needed. And cloud capabilities are relatively easy to procure because customers can sign up with a credit card—via an online portal—and start using services within minutes. This is the public cloud delivery model of Amazon and Google, among others.

Public clouds, however, can have a “not-so-silver lining,” with documented concerns over security, privacy, availability, data loss and latency issues. Organizations wishing to mitigate risks associated with storing and analyzing sensitive data in public clouds are increasingly turning to private clouds.

Public cloud solutions generally satisfy user expectations for applications like sales force automation or marketing campaign management. However, data warehousing requirements such as high availability, mixed workload management, near real-time data loads and complex query execution are not easily managed or deployed using public cloud computing models. By contrast, private clouds for data warehousing offer the higher performance, better security and predictable service levels expected by today’s business users. Read more

Liberal Arts or Business Degree? Or Both?

Business schools are often criticized for teaching aspiring executives what to think, rather than how to think. That’s why some professors suggest that liberal arts education is a superior choice to business school. But this argument misses the mark. What’s needed to operate in today’s complex world, is both business knowledge and a Socratic “mode of inquiry” taught through a liberal arts education.

As quoted in the April 25, 2011 issue of Financial Times, dean of IE Business School David Bach laments business students are taught what professors think they should know and then sent to conquer the world. But Bach says more than assembling and analyzing facts, students need the ability to ask the right questions. Bach states one way to reach this goal is through a liberal arts education where students gain “a way of looking at the world, a mode of inquiry” instead of time tested memorize and regurgitate approaches. Bach claims we’d be better off spending time training executives to make connections, problem solve and communicate—all skills the liberal arts education seems to afford.

In fact, as Bach alludes, executives need both business acumen and “different perspectives” in order to make sense of the world. As executives swim in deep seas of too much data, increasingly they will require content and facts parsed in manageable chunks (perhaps via MapReduce or other type of analytic engine) to help provide distilled fodder for overwhelmed and overworked human brains. Then, analytical tools can help discern correlations and patterns not clearly evident and also highlight information of critical importance requiring further drill-down analysis.

In addition to algorithmic engines and other technologies, to succeed in an unpredictable and complex global environment, executives will need to pair business acumen with modes of inquiry. Today, business managers can take data and information that tools have sorted and prioritized and then utilize these findings to challenge existing assumptions, seek new and different approaches, and apply creative thinking to tasks at hand.

A cartoon from the Financial Times shows two executives having a conversation with one quipping; “I left business school knowing all the answers. They didn’t tell me the questions would change.”

Indeed, in dynamic global economy, the questions, much less the right answers change every day, and sometimes in microseconds. Business and economic facts are important, but what’s really necessary are technological engines to help sort, order and then analyze the monstrous data flood that invades our daily lives.

The use of technology to harness the big data flood coupled with a questioning mind can then help make sense of the world today, and also intelligently predict with a high degree of probability what’s coming next.

Fight Back Against Black Swan Fatigue

In today’s lean, just-in-time, and over optimized world, it’s not uncommon for executives roll their eyes when the term “Black Swan” is brought up in risk management discussions.  That’s because even though preparing for extreme events makes logical sense, there’s also a cost associated with redundancy and robust disaster planning.  In addition, no one is ever judged a hero for saving the company from what never (or is never supposed to) happen.  Business executives must fight back against Black Swan fatigue, because in today’s interconnected and highly correlated world, the next extreme event could be the one that shoves your company off a cliff.

Image Courtesy of Flickr

When it comes to preparing for low probability but high impact events (i.e. Black Swans), the sad truth is most business executives will do nothing.  Why? Nassim Taleb, author of the Black Swan, explains; “It is difficult to motivate people in the prevention of Black Swans. Prevention is not easily perceived, measured, or rewarded; it is generally a silent and thankless activity. History books do not account forheroic preventive measures.”

Taleb is right. No one will be labeled a hero for keeping extra inventory on hand. No one will be characterized a hero for divvying orders among various suppliers just in case the favored and most cost effective supplier goes belly up. And spending money on strategy and risk management consultants to disaster and scenario plan for worst case developments? Forget about it. These are all just costs, and cannot be afforded in today’s bottom line economy, right?

Someone wise once said that risk management is much like insurance. You hate to spend money on it, but you’re darn glad you have it when all hell breaks loose.

But wait you say, don’t most business executives plan for disaster? Perhaps, but there’s a difference between hiring a consultant to produce a disaster planning report which promptly collects dust, and actually preparing for and assuming extreme events will occur as part of your overall business plan.  And even when managers believe they’re prepared for worst case events, sometimes it’s not enough—with potentially horrific consequences.

As detailed in the March 27, 2011 issue of the Financial Times, executives at Tepco’s Fukushima Daiichi plant were prepared for earthquake. In fact, they were also prepared for tsunami—having built a seawall 20 feet tall. What they did not expect is that the March 11, magnitude 9.0 earthquake would cause a tsunami wave 40 feet tall! The tsunami promptly washed away the sea wall and also the diesel powered generators cooling the spent nuclear fuel rods housed at Fukushima.

Executives at Fukushima had planned for disaster. They had built a 20 foot seawall. They had redundancy with backup generators in case the cooling system failed. And the nuclear plant powered down once the 9.0 magnitude earthquake hit. Everything worked as planned. But they were not prepared for the “unthinkable” extreme event.

Predictive modeling based on historical data will only take you so far. Even extrapolating with Bayes isn’t going to be of much use for “unknown, unknowns”.  As business managers we must fight Black Swan lethargy, especially when all oars in the boat are rolling towards lands of “optimization” and “cost effectiveness”. As managers, we must continue to sound the alarm, even though probability of the extreme event is of the smallest percentages.

Play up the risk of future Black Swans and then prepare for the extreme event. Here’s to hoping you’ll never be proved right.

How They Fit Together: Bell Curves, Bayes and Black Swans

Probability is defined as the possibility, chance or odds of likelihood that a certain event or occurrence will take place now or in the future.  In a world where business managers like to “know the odds”, how does probabilistic thinking (Frequentism and Bayesian) mesh with extreme events (i.e. Black Swans) that just cannot be predicted?

Image Courtesy of Wikipedia

Statisticians lament how few business managers think probabilistically. In a world awash with data, statisticians claim there are few reasons to not have a decent amount of objective data for decision making. However, there are some events for which there are no data (they haven’t occurred yet), and there are other events that could happen outside the scope of what we think is possible.

The best quote to sum up this framework for decision making comes from the former US Defense secretary Donald Rumsfeld in February 2002:

“There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – there are things we do not know we don’t know.”

Breaking this statement down, it appears Mr. Rumsfeld is speaking about Frequentism, subjective probability (Bayes) and those rare but extreme events coined by Nassim Taleb as “Black Swans”.

Author Sharon Bertsch McGrayne elucidates the first two types of probabilistic reasoning in her book “The Theory That Would Not Die”.  Frequentism (conventional statistics), she says, relies on measuring the relative frequency of an event that can be repeated time and again under the same conditions. This is the world of p-values, bell curves, coin flips, casinos and actuaries where data driven decision making is objective based on sampling or computations of large data sets.

The greater part of McGrayne’s tome concentrates on defining Bayesian Inference, or subjective probability also known as a “measure of belief”. Bayes, she says, allows making of predications with no prior information at all (no frequency of events).With Bayes, one makes an educated guess, and then keeps refining that guess based on new information, thus updating and revising the probabilities, and getting “closer to certitude.”

Getting back to Rumsfeld’s quote, Rumsfeld seems to be saying we can guess the probability of  “known knowns” because they’ve happened before and we have frequent data to support objective reasoning. These “known knowns” are Nassim Taleb’s White Swans. There are also “known unknowns” or things that have never happened before, but have entered our imaginations as possible events (Taleb’s Grey Swans). We still need probability to discern “the odds” of that event (e.g. dirty nuclear bomb in Los Angeles), so Bayes is helpful because we can infer subjective probabilities or “the possible value of unknowns” from similar situations tangential to our own predicament.

Lastly, there are “unknown unknowns”, or things we haven’t even dreamed about (Taleb’s Black Swan).  Dr. Nassim Nicholas Taleb labels this “the fourth quadrant” where probability theory has no answers.  What’s an illustration of an “unknown unknown”? Dr. Taleb gives us an example of the invention of the wheel, because no one had even though or dreamed of a wheel until it was actually invented. The “unknown unknown” is unpredictable, because—like the wheel—had it been conceived by someone, it would have been already invented.

Rumsfeld’s quote gives business managers a framework for thinking probabilistically. There are “known knowns” for which Frequentism works best, “unknown knowns” for which Bayesian Inference is the best fit, and there is a realm of “unknown unknowns” where statistics falls short, where there can be no predictions. This area outside the boundary of statistics is the most dangerous area, says Dr. Taleb, because extreme events in this sector usually carry large impacts.

This column has been an attempt to provide a decision making framework for how Frequentism, Bayes and Black Swans fit together—by using Donald Rumsfeld’s quote.

What say you, can you improve upon this framework?

Digital Design: Instagram and the Pursuit of Beautiful New Things

For some Silicon Valley companies, the mantra is: “Slap code together, design a quick user interface, and get it to market.” And above all, as Mark Zuckerberg would remind us, “Keep shipping.”

In the very near future, such talk will probably give Instagram founder Kevin Systrom heartburn. That’s because Systrom thinks designing software code should be an inspired effort, just like the delectable creations of top chefs around the world.

When it comes to digital products, the idea that form follows function is being tossed out the window. As a New York Times article describes, it’s no longer necessary to simply examine a digital device or application to discern what it does. But that doesn’t mean there’s not a sense of beauty and simplicity necessary for digital products. In fact, the goal for software engineers designing best-in-class digital products is to build instinctive and intuitive applications and devices.

Perhaps that’s why Kevin Systrom’s Instagram was worth one billion dollars to Facebook. As noted in a Financial Times article titled, “The Aesthete of Silicon Valley Who Makes Photos Beautiful,” Systrom is obsessed with building things that are inherently beautiful.  Systrom says great chefs who painstakingly labor over their creations and add high-quality ingredients despite potentially higher costs inspire him. “The things we’re creating solve a need,” he says, “but in the meantime, they should be visually stimulating and fun to interact with.”

An obsessive personality, Systrom would likely have been a better disciple of Steve Jobs rather than of his new boss, Mark Zuckerberg. “Kevin notices the details that most of us don’t notice,” says Clara Shih, a friend of Systrom. “He’s a slow walker and lets himself get distracted. He takes time not just to smell the roses but to analyze the roses and think about the detail.”

Technological advances are coming so fast that it takes us a long time to assimilate all the benefits of each product and service. For example, software companies are notorious for rolling out hundreds of new features in each release, only to have customers use very few.

As our digital lives get inevitably more complicated, there will be an insatiable customer requirement for simplicity and aesthetic beauty. And as user interfaces get more cluttered, clarity in purpose and design will drive market share (as Google can attest). Indeed, the elegance of a single search box—unencumbered by advertising—is one reason Google has the lion’s share of search requests.

Instead of focusing on additional features that users will probably never use, digital companies of all stripes perhaps should aim for a crisper design and greater ease of use.

Ultimately, making digital products easier to use takes a long time in discerning and studying user behaviors, capturing those learnings, and using them to design better and more intuitive devices and interfaces. However, much like how top culinary chefs take the time to use better ingredients and obsess over the tiniest details in presenting their creations, digital creators will need to endeavor to stress the small stuff.

For digital consumers, naturalness, comfort, and fit will matter more in the near future. And thoughtfulness in terms of design and ease of use will be the next generation of differentiation for digital companies.

Finally, there’s been a bit of stress for Instagram users now that Facebook has purchased the company. Despite Facebook saying it will run the company independently, plenty of users fret that Instagram will lose its touch. However, wouldn’t it be something if Systrom didn’t end up leaving Facebook after all and instead became the company’s Jonathan Ive? It could end up being a “magical” pairing.