Extreme Redundancy – Don’t Leave Home Without It!

Visa’s command and control center—somewhere on the east coast of the United States—is a case study in redundancy with intense security, backups (for everything), and failover processes for system failures. And while most businesses don’t require these same levels of high availability for their own information networks and data centers, Visa can certainly teach a course on risk management to those who don’t believe such redundancy is necessary.

In the past four years, the unthinkable is happening more often than predicted. From the global stock market meltdown of 2008, to the Japanese tsunami disaster at Fukushima, to flooding in Thailand, and many other prominent examples, there’s Black Swans aplenty without signs of abatement.

Most of these events were near impossible to predict, but it’s entirely within the realm of risk management to prepare for heavy tails. The key to countering effects from extreme outlier events, says Black Swan author Nassim Taleb, is to build redundancy into business processes. He counsels companies, enterprises and even countries to “avoid optimization (and) learn to love redundancy.”

And while it may be a progressive example, Visa provides a well documented case study into managing fourth quadrant risk.  A Fast Company article titled, “Visa is Ready for Anything” provides a rare view in Visa’s data centers where 150 million daily transactions are processed.  Taking a page from NASA’s Mission Control, this data center monitors Visa network security, availability and capacity on a 24x7x365 basis. And meeting the definition of a “Tier 4” data center; “every major system—mainframes, air conditioners batteries etc, has a backup.”  With backups for backups and disaster planning in place, this data center is designed to withstand attacks from all sorts of events – terrorism, hackers, and even natural disasters such as earthquake and/or tornado.

While it’s true that Black Swans are about “unknown, unknowns”, and preparation for every type of risk is not only unrealistic but also too expensive, there is much companies can do to build more redundancy into daily operations. Redundancy is especially relevant as “interlocking fragility” of global communication, supply chain, and financial networks means correlations move to one with increasing frequency.

Today’s business culture celebrates “optimization” for every process; in essence living as near the edge as possible without falling off the cliff.  That’s a dangerous strategy for today’s interconnected world, especially because extreme events can take networks offline for hours, days and even weeks, costing companies tens of millions in lost revenues.

Critics of redundancy argue that robustness is too expensive. But Dr. Taleb counters; “Redundancy is like (being) long on an option, you certainly pay for it but it may be necessary for survival.”

Black Swan Alert: Low Tech Links Devastate High Tech Supply Chains

Find your supply chain best practices checklist. Just-in-time processes? Check.  Optimum sourcing strategies? Check. Lean logistics? Check. Best-in-class technologies? Check.  Black swan avoidance? Uh-oh.

Bangkok, Thailand is under water.  In perhaps the world’s worst flooding to devastate such a large city in modern times, Bangkok is home to ten million citizens and has an additional ten million people outside city limits.  The Chao Praya river drives down the middle of Bangkok and is supported by dikes and drainage canals for rainy seasons. But this month, the river is eight feet above normal, and while dikes have not failed, flooding has caused most residents to leave the city.

The human tragedy of such a flood aside, what’s occurring in Thailand is on just about every global CEO’s radar screen. And that’s because Bangkok is a crucial link in many high tech supply chains.

David Pilling of the Financial Times notes that companies such as Mazda, Toyota and Toshiba all have suppliers and factories in Thailand. And Honda’s Thai assembly plant, which traditionally manufactures 250,000 cars a year, has been shut down for three weeks.   With such interconnected and complex supply chains, sometimes spanning many countries, Pilling says these links are “prone to strain, particularly when paired with the just-in-time practices pioneered by Japan.”

Many supply chain gurus push companies to optimize inventory practices, adopt just-in-time strategies, and reduce redundancy wherever possible. And in “normal times” this approach makes sense. However, when disaster strikes, manufacturers are discovering their thin margins for error are in fact, leaner than they should be.

Nassim Taleb, author of The Black Swan, and professor of Risk Engineering at New York University is no stranger to interlocking fragility. He says most business professionals assume we live in a world of mild randomness, where events don’t stray far from the mean. Taleb argues the opposite, that in fact, because of tightly coupled financial and consumer markets (much less recurring natural disasters) we live in periods of wild randomness where small probability events carry large impacts (i.e. Black Swans).  These extreme events—Thai flooding or Fukushima disaster, for example, are shutting down entire supply chains for weeks and even months, causing hundreds of millions in lost profit opportunities.

A solution proposed by Dr. Taleb and others is to build a robust system (in this case supply chain) with redundancies and disaster recovery processes to properly manage extreme event risk. And while some experts argue redundancy adds costs to already paper thin margins, there are surely costs to not supplying marketswith needed product because an unforeseen event has silenced your supply chain.

Bob Lutz, former vice chairman of General Motors says it best; “Running your procurement purely on a short term, point in time, cost minimization model is like shopping for rock bottom home insurance. It looks real smart until your house burns down.”  He goes on to say; “What happens when ‘just in time’ is ‘just plain late’”?

Good question and one which many global CEOs are still struggling to answer.

USAF Prevents Insomnia with Geospatial

What causes sleepless nights? For sure, too much caffeine can keep your eyelids open, however it’s more likely that worry and anxiety are the source. Fortunately, insomnia’s not the norm for United States Air Force (USAF) general officers and other government officials. That’s because with geospatial capabilities the USAF knows where every aircraft, piece of equipment and part is located and where it’s been—anywhere in the world.

Image Courtesy of Google Maps

Geospatial (also known as location intelligence) is a technology that allows enterprises to store, process and consume geographically based data. Geospatial data can be points (longitude, latitude), lines (a series of points), or even polygons representing a defined boundary. With geospatial, an enterprise can measure distances between two objects, or identify objects within, touching or traversing a specified perimeter. And while approximately 80% of corporate data has a location reference, there are plenty of enterprises not taking advantage of this valuable location information.

At Teradata Partners conference 2011 in San Diego, California, speakers Tim Cotton and Teal Walker presented how geospatial capabilities benefit the USAF. Specifically, leveraging data integrated from over one-hundred sources into Teradata database and Google Maps for visualization purposes, the USAF provides its users trusted information for inventory control including drill down capabilities to the part and supply level. And with geospatial, officials can visually examine where a particular asset has been throughout its lifecycle.

Teradata and Google Maps also help the USAF visually monitor “exceptions” via alerts—in real time—to track movement of materials, vehicles, commodities and assets. Now any aircraft or equipment going to the wrong place can be identified quickly and rerouted.  And lastly with geospatial capabilities, the USAF can easily perform “proximity analysis” to immediately discover assets nearby and available to maintain mission support.

Geospatial enables “big picture” location analysis. The USAF knows where its assets are located at all times, and can direct them quickly should the need arise. Important operational details are captured and readily available via this geospatial solution, allowing government officials to clear their minds and get a good night’s sleep.

The USAF can pinpoint and direct its most productive assets at a moment’s notice. Shouldn’t your business have this same advantage?

Black Swans Causing a Rethink on Global Supply Chains?

A fair percentage of academicians and corporate supply chain experts agree that running supply chains “lean and mean” with just in time processes is the optimum approach. However, Black Swans (low probability events with high impact) such as the March 11, 2011 Sendai earthquake and subsequent tsunami have companies rethinking how they source to deliver the products and services critical to success.

The 8.9 magnitude earthquake and tsunami that ravaged Japan’s coast and cascading Fukushima nuclear calamity have severely tested the links in today’s global supply chains. And while it’s true that some Japanese electronics and automobile manufacturers continue to struggle with factory shutdowns, most global supply chains weathered the storm quite well. However as supply chains arguably become more complex, executives should reconsider how man-made and natural disasters can affect processes in delivering goods and services to market.

A Financial Times article titled “Industry Left High and Dry” cites how some companies were not prepared for the repercussions of Japanese factory slowdowns or shutdowns. These enterprises relied too much on Japanese factories to produce critical electronic components, and others simply lacked contingency disaster planning. Even more disturbing, Barry Tarnef, a risk manager at US insurance group Chubb says; “Many manufacturers (lacked) the information needed to mobilize alternative arrangements quickly (when their supply chains were interrupted).

One of the themes that I have highlighted over the past 2-3 years is with the global integration of trade, capital, information and labor over the past decade; few people understand that our world is more tightly connected than ever before. In a highly inter-dependent environment, small mistakes can have large ramifications. And even larger events – such as tsunami, volcanic ash and the like can cause an earnings miss or worse depending on the severity and criticality of the event.

That’s why companies are starting to re-consider their global supply chain processes. This means structuring alternative supply networks, adding regional suppliers (instead of relying solely on one country), and doing a better job of setting up an IT infrastructure capable of linking suppliers and providing supply chain visibility from “field to fork”.

Black Swan author Nassim Taleb has long counseled companies to consider concepts of robustness and redundancy to build flexibility in response to adverse events. However, these considerations surely add additional costs, and many companies from retailers to manufacturers are already relying on paper thin margins.

One thing is for certain, there are no excuses for ignorance of how your supply chain processes work – or don’t work. The supply chain that weathers today’s storm may be wholly inadequate for tomorrows.

Question:

* For supply chains, should companies continue to “go lean” or build in redundancy? What are your thoughts?

Supply Chain Traceability – What’s in Your T-Shirt?

Does it concern you that cotton from a t-shirt originated in Uzbekistan? You might care if reports are true that cotton harvested from this country came from forced or child labor. In fact, consumers are increasingly asking manufacturers and retailers to ensure goods and services are produced with sustainable and ethical business practices. With goods processed between countries and multiple supplier touches, traceability in the supply chain is more important than ever.

Consumers are becoming more and more interested in the make-up of products they’re buying. The ability to trace products through the entire supply chain—whether from field to fork or source to consumer—is becoming a competitive weapon for some companies, especially if they can authenticate then promote goods produced with sustainable business practices.

A Financial Times article cites that ethical spending has increased significantly in the past 20 years. As an example in the United Kingdom, “The amount of spending and investment influenced by ethical considerations almost doubled between 1999 and 2008 to reach £36 billion.” The FT article mentions the growth of fair trade and certified organic products as part of this growth trend. And fellow DailyFix author, Ted Mininni, says these types of goods, “are increasingly being added to retail assortments”—and in growing numbers!

However, one of the major challenges for both retailers and manufacturers alike is authenticating ethically sourced products. There’s more to the process than taking the supplier’s word for it.

In the case of organic bananas, the FT article notes, “the food company Dole labels each of its organic bananas with a three-digit number that, when entered on its website, reveals details of the farm where that banana is grown.” Identifying where a banana is sourced, however, is simple compared to such products as sweaters or t-shirts that pass through multiple suppliers and countries.

In “The Elephant and the Dragon,” author Robyn Meredith confirms that products often “zigzag” through the global supply chain from factory to factory. “A cheap toy may be assembled by parts from 12 different factories,” she says. And something as sophisticated as an automobile might contain five to seven thousand parts.

Fortunately for marketers, the challenge is not beyond the capabilities of today’s technologies. Some progressive companies are engaging in a purposeful effort to build effective policies (including auditing), technologies (supply chain analytics and infrastructure) and processes to track and monitor the extended supply chain.

Of course, implementing a data-driven supply chain infrastructure is only half the battle—supply chain managers, operations personnel and marketers must learn how to use it! Marketers, working alongside operations, will need training on the various tools and systems used to access data for reporting and query purposes. And marketers may also choose to make supply chain data available directly to consumers—similar to Dole’s web portal—thus enabling them to verify product origins for themselves!

Ethical sourcing, trade, manufacturing and retailing will continue to be a hot button for consumers. However, as seen from this article, jumping into this marketplace requires much more than just fancy signage and/or promotion. A real commitment to corporate responsibility and sustainable practices must be much more than lip service; it involves significant investment in people, processes, technology, and strategy. As seen from the complexity in supply chain traceability alone, it’s definitely not an effort a company should take lightly.

Questions:

• Does it matter to you how a product is made? Are you interested in the origins of the products and services you consume?

• The Financial Times article says that companies should make information about their supply chain public—or consumers will do it for them. Do you agree with this statement?