Five Data Warehouse Articles: Get More for Your Money

At the risk of falling into the well of self-aggrandizement, I’d like to offer links to four data warehousing articles, and a forthcoming article for October 2009.

• Above and Beyond (October 2009) – Managed Services helps optimize customer data warehouse environments
• Peak Performance – Six best practices help maintain a healthy active data warehouse
• Strike it Rich – Application tuning helps companies save money through query optimization
• Within Your Reach – Active Enterprise Intelligence extends the reach of data-driven decision making throughout the enterprise to arm users and systems with the right information and analyses
• So You Don’t Need a Methodology? Creating a data warehousing “blockbuster” begins with the right methodology

Consumer Choice: Less is More

Many marketers believe that innovation and competitive differentiation arise from giving customers more choices and options. But through the strategy of “offering more choice”, marketers may actually end up increasing complexity, costs and causing customers “mental fatigue.” Is there a better way to win over customers?

A typical big box retailer carries 50,000+ stock keeping units (SKUs). Retirement plans carry hundreds if not thousands of investment options. A website designer offers customers 216 color palette options for a home-page.

With customers drowning in “choice” some companies are finding it easier to meet customer needs by simplifying—portfolios, products and services.

Case in point, the Wall Street Journal published an article, “Ford Eyes More Cuts as Recovery Advances”, detailing some of the decisions that Ford Motor Company has made reduce the “mind-boggling level of vehicle customization, which jacked up costs.”

For example, Allan Mulally, CEO of Ford joked that until recently, the Lincoln Navigator offered 128 options on its console alone. “You know what 128-factorial is — it’s a lot of combinations,” he said. The article points out the real answer: 3.85620482 x 10 to the 215th power.

Ford is finding cost savings and efficiencies in getting back to basics, streamlining operations, and reducing the complexities of the products they offer customers.

Indeed too many choices can cause our customers to experience anxiety and mental exhaustion.

According to the April 2007 issue of Journal of Personality and Social Psychology, people have difficulty staying focused enough to complete projects when presented with too many choices.

Researchers conducted seven experiments involving 328 participants and 58 consumers at a shopping mall.

One group of participants was asked to consider a multitude of options (which products to buy, what classes and coursework to take etc) and then actually make decisions. The other group was allowed to consider options but wasn’t instructed to actually make a choice.

Both groups were then asked to do various tasks. The group involved in decision making (choosing from the options) had difficulty staying “on task and maintaining behaviors aimed at reaching a goal.”

According to the research, mental fatigue results from not merely considering a set of options, but in actually prioritizing and choosing among the options.

Intrinsically this makes sense. Suppose you asked me to list, off the top of my head, ten cars that I’d like to purchase assuming I had the means to do so. Not only could I come up with ten, I bet I could rattle off twenty!

However, suppose you said to me, “Now of those ten cars, I’m going to give you one, so you need to choose.”

Suddenly I really have to think. What brand, what options do I want in the car, seat color, interior color, leather or cloth, V6 or V4, two or four door etc? My brain really has to start working!

Kathleen D. Vohls, the study’s lead author concluded that making choices depletes a precious resource in the human mind and causes mental exhaustion. “There is a significant shift in the mental programming that is made at the time of choosing,” she says. “Simply the act of choosing can cause mental fatigue.”

So it’s not just the pondering of choices, it’s the actual prioritizing and choosing that mentally wears us down.

We live in an era of plenty. Starbucks, for example, says they have 87,000 different ways to get you a drink.

As marketers, we need to help our companies focus and prioritize on the things that matter most to our customers.

Since marketing is responsible for the “voice of the customer”, we must help steer R&D, product management, finance, operations and other corporate functions towards adding (or subtracting) features/functionality from our products and services that will actually make a competitive difference.

Sometimes, less is more —especially when it comes to “choice”.

Better Messaging: Think Spacing Effect and Forgetting Curve

As marketers we often struggle with getting customers to pay attention to our marketing messages, much less remember them.

A specialized learning technique—the “spacing effect”—is helping people all across the world assimilate more languages, facts, and retain gobs of information. Is this learning technique applicable to improving the efficacy of our marketing campaigns?  read original column

Should Marketing Executives Skip the IT Department?

As marketers become more tech-savvy, the demands they place on corporate information technology (IT) departments often increase. Unable and reluctant to meet the demands of marketing, IT sometimes turns a deaf ear to change and budget requests causing marketers to go it alone. However, going it alone—without IT—can create headaches, internal conflict, and strip marketing of budget that should be used for marketing programs. Should “marketing” be working closer with IT, or taking control of its own destiny?

With the ubiquity of smart phones (including iPhone and Blackberry), social networking platforms and marketing applications, marketers have more technology choices than ever before to help them communicate and interact with customers.

And by adding software-as-a-service (SaaS), application service providers (ASPs), and marketing agencies to the mix—marketers can effectively get their needs met—often without the help of their local IT professionals. But is doing an “end-run” around IT appropriate—or recommended?

CIOs are starting to realize that business (in this instance marketing) executives are becoming more, and more tech savvy, and that either IT should help enable and drive the business or get out of the way.

An article in Information Week, “Don’t Let Tech-Savvy Business Execs Do an End-Run Around IT,” gives CIOs tips on tactics to regain control of the technologies within an enterprise and work more closely with business users.

Suggestions for IT managers (paraphrased) include:

• Making it corporate policy that all IT contracts must be co-signed by IT
• Making closer connections to business departments
• Keeping an open mind
• Letting business users “test” their ideas
• Providing technology choices—not mandates
• Communicating clearly and concisely

In the article, some CIOs lament that business users aren’t as tech savvy as they should be (translate—they don’t know what they want much less what they need). And CIOs often see business users buying technology that could help more than just their own departments if widely deployed.

Other CIOs quoted in the article grow weary of cleaning up the messes that business users make when they try and tackle enterprise technologies.

For the record, I strongly support working closely, hand-in-hand, with your local CIO on technology-driven initiatives.

Arguably, your CIO has an enterprise-wide view, and can help ensure your technology investments “fit” and integrate with other corporate systems.

CIOs can also assist with “re-use” of technologies, defraying some of the costs to other business units. In addition, in many instances CIOs have significantly larger budgets and might be able to fund technology investments so marketers don’t have to spend meager marketing budgets on applications or infrastructure.

In addition, as detailed in a MarketingProfs article, I believe that to survive and capitalize on the exponential trends of technology and data growth, CIOs and CMOs will have to align strategies, pool resources, communicate more effectively, and find common ground wherever possible.

However, taking off my Pollyanna glasses, I realize that in some companies, due to politics, budgets, organizational silos or fractious personalities—avoiding an end-run around IT may not always be possible.

Questions:
• Do you work effectively with your IT department on initiatives that drive the business?
• Have you found instances where it’s better to go-it-alone?
• Some CIOs lament that marketers aren’t tech savvy—do you know as much about technology as you should?

Why Capacity Management Matters to Business Executives

In a challenging global slowdown, the world seems awash in capacity. Scans of major business publications show airlines reducing flights, companies furloughing or firing employees, and manufacturers closing plants. If you agree that it appears there is more unused capacity than demand, why should capacity management matter?

It would seem in the “Great Recession,” capacity planning and management should be a minimal consideration. In fact, capacity utilization for many industries is at an all time low.

For example, from January 2008 to January 2009, according to the Financial Times, the demand for automobiles in the United States fell from 15.9 million to 9.6 million per year. And Wall Street Journal reports Federal Reserve Chairman Ben Bernanke told the House Budget Committee recently, “The slack in resource utilization remains sizable”.

As companies attempt to cope with a “new normal,” painful restructuring processes have included reducing “capacity” in human resources, plants and equipment, information technology, number of brands, distribution channels, and even debt covenants. All this restructuring is intended to pare down capabilities to what is perceived as a new reality in market conditions.

Indeed, observing macro-economic conditions, it’s tempting to write off “growth.” However, “growth” is far from dead.

Take for example, the exponential growth trends of Facebook and Twitter. In January 2009, Facebook touted its 150 millionth user, and in May 2009 surpassed 225 million users! One site projects Facebook to have 300 million users by the end of the year. Twitter’s growth has also been phenomenal—audiences grew 40% in just 30 days (March-April 2009).

In fact, “growth” exists (often exponentially) in areas such as data volumes, populations, energy usage, Moore’s Law, GDPs of select countries (India, China etc), education expenditures, and unfortunately—state and national debts!

Growth also can be found in micro-segments and categories such as increases in market share of private label brands vs. national brands on grocery store shelves, or Apple’s share of the smartphone market. Once our eyes are opened to growth trends, it’s quite easy to see signs of expansion everywhere!

The ability to meet the needs of your customers now and in the future is a critical function of any business. That’s what capacity management is all about. Spikes in demand could mean that your company is leaving money on the table and/or failing to meet customer needs. Need proof? For customer reaction, simply perform a web search on keywords “Twitter down time” or “Twitter outage” and you’ll gain evidence of how important capacity management really is.

Indeed, capacity management isn’t a one-time, annual event. It should be a continual process of making sure your business can scale up or down to meet customer needs. With a thumb on the pulse of demand, marketers have a responsibility to help establish a well documented capacity plan and process that considers future business requests.

Sound like simple, common sense, right? Properly predicting demand is anything but easy. Considerations must include a clean and accurate set of historical data, an analytical infrastructure to compute and analyze data, an understanding of the current state of the business and its capabilities, future growth projections based on applicable trends, and then a gap analysis of what it would take to scale based on various “what-if” scenarios.

Capacity management is all about reducing surprises. Take a good, hard look at your business. What’s growing? Something surely is.

What marketing campaigns are you preparing? What happens—for goodness sake—if they’re too successful and demand exceeds available supply? McDonald’s in India had to scale back marketing campaigns for Chicken McNuggets because they couldn’t keep up with demand. Good marketing is making promises your company CAN deliver.

Can you accurately predict if and when you’ll run out of resources to meet customer needs? Can you afford not to properly manage “capacity”?

Questions:

• There appears to be a glut of capacity worldwide (i.e. shipping, telecommunications, manufacturing etc.). Should marketers be concerned with the concept of capacity management?
• What are the ramifications of getting capacity management wrong?
• Businesses are adding flexibility to meet spikes in demand through vehicles like cloud computing, temporary labor and outsourcing. Can you think of others?
• Suppose “capacity management” is built into the function of an annual strategic planning exercise. What might be a pitfall of this approach?

Mathematics Invades the Marketing Kingdom

Think marketing doesn’t have much to do with mathematics?

Mathematics  is giving some companies a competitive edge in better understanding customers. Indeed, companies across all industries are now capturing data and creating rich profiles of customers to “predict” their wants, needs and future desires.  Mathematics has left the ivory tower of academia for a marketing department near you. Are you ready for this massive paradigm shift?

Let’s be clear as one who has worked for many global consulting companies,  I hate the phrase, “paradigm shift”. The words are close to meaningless due to overuse.

However, in this rare instance, where the world of mathematics is invading the marketing kingdom, it makes sense to emphasize a new way of thinking that is radically changing the way marketers do business.

Marketers have always wanted to know more about customers—after all, better segmentation and targeting of a customer base helps improve marketing ROI and ultimately increases satisfaction as customers are not bombarded with irrelevant offers.

Fortunately for marketers, advances in technology (both applications and infrastructure) have made it easier to capture, manage and analyze data so as to piece together a more complete picture of customer behavior and of enterprise operations.

Case in point, an article from Business Week titled, “Math Will Rock Your World”, highlights companies such as Google, Aetna, Harrah’s and others that are using mathematics via analytical applications to sort out “swelling oceans of data” and mine data for insights to better understand customers.

While arguably a bit dated, the Business Week article showcases how companies are using customer data to build profiles and formulate models of both customers and employees that they believe will allow them to simulate and predict how to, “sell us things, steer us clear of diseases, and ramp up (employee) productivity.”

Other examples in the article show how companies are using advanced algorithms to make sense of unstructured data (emails, documents, call center notes), and optimize online advertising campaigns through the refinement and selection of keywords for search.

Using mathematics to better understand customers is serious business—just ask Netflix.

A Wired magazine article discusses how this online movie rental company offered a $1 million dollar prize to any one person or team that can improve its movie recommendation algorithm by 10%. In fact, according to Netflix’s leaderboard, a team from Bellkor just won the prize!

By opening access to one of the largest data sets available of online behavior—100 million customer movie ratings—Netflix has “crowdsourced” improvements to Cinematch, its engine that essentially recommends, “If you liked this movie, you’ll also like this one.”

However, even though it has taken over two years for a team to finally win the Netflix prize, minor improvements along the way to Cinematch have helped Netflix utilize more of its DVD inventory and improved customer loyalty as subscribers find movies of interest that perhaps they might have previously overlooked.

In another article, “Guessing the Online Customer’s Next Want”, Barney’s New York is mentioned as a company that’s seen dramatic marketing ROI improvement from using sophisticated analytical applications based on complex mathematics.

Through the use of technology, Barney’s is able to collect and analyze the online behavior of its customers and then craft smarter and more appropriate responses to interested audiences.

For example, the article notes, “An e-mail message announcing sales might go to those Web site visitors who had purchased certain products or types of products in the past, but who had done so only when the items were on sale. In the simplest terms, if someone buys only when something is on sale, but never buys anything in December, then the e-mail sale flier might not be sent to that customer in December.”

Just as in the early 1980s, when the financial industry was upended by the flight of quants from academia to Wall Street, marketers are starting to reap the brainpower of mathematicians, physicists and others as they codify their expertise and knowledge into sophisticated information technology systems and analytical applications. These innovative systems are helping marketers leverage information to better connect with customers and drive the business forward.

Paradigm shift? Absolutely. The world of marketing will likely never be the same again.

Questions:
• Are you seeing these trends in your particular industry? If so, how so?
• The companies mentioned above are starting to treat data as one of their most valuable assets. Is your company on that path?
• Are you concerned with the potential “dark side” of simulating and modeling customer behavior—i.e. privacy issues?
• What skill sets will marketers need in the future to be able to compete in this new world of mathematics and marketing?

I’d love to hear from you!

Data Visualization – One City at a Time

Ever wondered what the “rhythm” of your city looks like? In cities like Rome and New York, aggregated real-time data from mobile providers is helping government officials monitor traffic flows, efficiently utilize transportation networks, and even plan for large-scale events helping to improve overall “citizen satisfaction.” Is real-time data visualization coming to a city near you?

The proliferation of mobile and GPS technologies (sometimes in the same handset), are making it possible for city planners, government officials, and even businesses to gain a pulse of the daily movements of entire populations.

An article in the Wall Street Journal, “Cellphone Data Track Our Migration Patterns”, June 10, 2008, mentions how mobile providers are allowing access of anonymized and aggregated location data to social scientists, physicists and urban planners.

The article notes, “More than 3.3 billion wireless-phone subscribers world-wide have, in effect, voluntarily adopted devices that record their daily movements in the same way satellite sensors monitor migrating birds, whales, bears and other wildlife.”

Indeed, network physicist Albert-Laszlo Barabasi says that since practically everyone has a mobile phone, “Everything we do leaves an electronic fingerprint somewhere.”

While the concept of tracking human movement raises the eyebrows of many privacy advocates, the benefits to exploratory analysis of GPS and mobile data—according to the article—are myriad including, “aid(ing) emergency relief efforts in natural disasters, as well as improving urban planning, public transportation and traffic control.”

The Wall Street Journal article also mentions another case study where MIT and Telecom Italia have teamed through Project “Real Time Rome” to help its citizens and officials make better decisions regarding resource utilization.

Overlaying telecommunications data and Google Earth allows Project “Real Time Rome” to dynamically reveal “the rhythm of the city” through six visualizations:

• Pulse – helps determine commuting patterns and patterns of use for transportation networks
• Connectivity—ensures public transportation is located near populations
• Flow—helps answer the question, “Where is traffic moving or flowing to?”
• Icons— helps answer the question, “Which landmarks in Rome attract more people”?
• Visitors—discovers where tourists congregate
• Gathering—during special events (i.e. Madonna concert), helps determine how people occupy and move through different parts of Rome

Sophisticated analytical applications and data warehousing technologies are helping “bring data to life” for governments, citizens and businesses. I’ve also thought of some other ways this mobile/GPS data could create advantages:

• Businesses could examine historical patterns of people movement to decide where to open their next store/branch
• Real estate agents (commercial and residential) could use the data to determine migration patterns over time
• Businesses could examine the data to determine staffing and inventory levels by day or even hour based on historical traffic patterns in their vicinity
• Variable pricing could be enacted for access to roads or public transportation based on peak-demand usage (ex: toll roads into a city could charge more for drivers during peak hours much like London is doing)
• Chambers of commerce and city officials could use the data to steer promotions/traffic towards a new downtown renovation, places of interest, less frequently visited tourist attractions.
• Special events—mid-city concerts for example—can be modeled based on historical data of traffic, pedestrian flow etc, to ensure future events are more accessible

Through the use of powerful data-visualization applications, government agencies, businesses and citizens are able to explore data to uncover mathematical patterns and connections to help improve the lives of everyone concerned.

So, the next time you visit a major city, hit all the tourist attractions in a timely fashion, avoid the crowds and notice that trains, buses, and taxis run on time and are conveniently located, remember it’s probably not an accident. Good service rarely is.

What are your thoughts?

Desperately Seeking Distinction: What’s Your Advice?

To compete for lucrative Wall Street or consulting jobs, students across the United States are joining on-campus investment clubs to help prepare them for real-world situations.

In fact, students in all disciplines are racing harder and faster than ever to differentiate themselves before they graduate. In a “race for distinction”—how should a student stand out?  read original column

In a Petabyte Age, Is Understanding Passé?

Analysts have estimated that the volume of data in enterprises of all sizes is doubling every two to three years. With the deluge of data, some companies are finding it makes more sense to discover and act upon patterns (i.e. customers who buy item X also buy item Y), rather than dig deeper and search for causation. In an age of cloud computing and “big data”—where correlation is often sufficient to gain business results—are we losing our thirst for knowledge and understanding?

Chris Anderson, editor of Wired Magazine and author of “The Long Tail” penned a provocative article in the July 2008 issue titled, “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete.”

Mr. Anderson makes the claim that in “the Petabyte Age”, it’s more important for companies (and the marketers within them) to identify and act upon correlation first, and worry about context later.

For instance he writes, “Google’s founding philosophy is that we don’t know why this page is better than that one: if the statistics of incoming links say it is, that’s good enough. No semantic or causal analysis is required.”

And dismissing many of the sciences that attempt to bring us understanding of the world around us, Mr. Anderson notes, “Who knows why people do what they do? The point is that they do it and we can track and measure it with unprecedented fidelity. With enough data, the numbers speak for themselves.”

As companies collect more data about their customers, competitors and macro/micro environments, Mr. Anderson makes the claim that our approach to science (hypothesize, model, test) is becoming obsolete.

Science looks for causation. Scientists hypothesize as to why something works/reacts/behaves like it does and then attempt to build a model to represent reality. The goal is to use the model to test and learn, thereby gaining an understanding of a particular phenomenon.

Modeling is not only confined to the realm of physicists and quants. In the field of marketing, managers often work alongside statisticians, to build models that help predict customer proclivities such as items they might like to buy, or identifying customers who might churn to a competitor. Models can be tested and refined and with enough tweaking, models generally get more accurate over time. The output of those models can be used to piece together more complete customer profiles.

However, in the Petabyte Age, Mr. Anderson claims, “There is now a better way. Petabytes allow us to say correlation is enough. We can stop looking at the models.”

Let’s suppose Mr. Anderson is on to something. Mr. Anderson makes the claim that since modeling is often a poor representation of reality and tends to over-simplify things, we’re much better off with less explanation and more action based on the identification of correlation.

In a sense, he says, in an age of massive data, we’re better off with fewer discoveries of knowledge and understanding.

I’m not sure I agree.

Peter Atkins, author of “Galileo’s Finger”, says it much better than I can: “With the rise of the computer and its ability to handle huge numerical calculations of the greatest intensity, we are seeing a shift from analysis to numerical computation. (This is dangerous because) resorting to numerical solution can distance us from understanding.”

It is true that building models is an imperfect science, and cannot in all instances be 100% accurate—it is after all just a model!

That said, models help us test our assumptions and verify our forecasts. They help us test what we think we know, with the ultimate goal of improving our decision making in real-life situations where marketing budgets and return on investment are on the line. Modeling helps us piece the world together, and look beyond the patterns towards discovering “why” things happen as they do. And models help us transform reams of raw data into intelligence thereby helping us predict outcomes with greater accuracy.

While focusing on correlation can help us make better decisions to a degree, philosophically I am concerned with marketers potentially losing our interest in truly understanding what makes our customers, prospects and partners tick. For me at least, I want to know more than “my customers do this, or they do that”; I want to know “why”!

In all fairness to Mr. Anderson, every coin has two sides. There is a fine line between too much pontification and too little action. Indeed, there are many instances where it doesn’t make sense to dig deeper in understanding—where it doesn’t matter “why”, only that a given solution produces results.

Chinese Brands Are Coming!

Chinese brands have come a long way since the days of Mao where coats and boots came in two colors and consisted of dubious quality. Rising in complexity and caliber, brands such as Haier and Chery are not content to rule the middle kingdom and instead seek to invade new markets.

Forget bringing your brand to China, are you prepared for the onslaught of Chinese brands in your home market? read original column