It Takes Courage to Compete on Analytics

Billy Beane’s “Moneyball” approach to developing and staffing a professional baseball team has come under intense scrutiny as long time major league scouts and analysts take delight in sub-par performances of the Oakland Athletics.  Beane however seems undaunted in using statistical analysis to undercover market place inefficiencies.  Indeed, it takes courage to solve problems in a whole new—perhaps heretical—manner. And while plenty of folks take pleasure in Beane’s recent comeuppance, there’s a good chance he’s already had the last laugh.

By now, most business professionals have either read Michael Lewis’ “Moneyball” or at the very least seen Brad Pitt’s rendition of Billy Beane in Hollywood’s version.  Beane’s claim to fame is that he had the courage to defy decades of “common knowledge” in assembling a winning baseball team.  For many years, winning teams were accustomed to paying top dollar for the best hitters and pitchers evidenced by metrics such as batting average and earned run average respectively.

However, in the small-town Oakland market, Beane didn’t have the luxury of affording a high payroll. And if he wanted to compete, he needed to find a different (and perhaps better) method of keeping up with the likes of the Boston Red Sox and New York Yankees.

Beane knew there were “inefficiencies” in baseball markets that could be exploited; he just had to discover those undervalued metrics.

Enlisting the help of his Harvard friend Paul Depodesta, Beane used statistical analysis to discover metrics that—in his opinion were more indicative of baseball success—such as on base percentage and slugging percentage for batters, or the ability to get “outs” for pitchers.  Then Beane went about acquiring a motley bunch of players that most teams discarded as near worthless, and positioned each player for success based on his analytical models.

And for a while it worked. With one of the league’s lowest payrolls the Oakland A’s competed with teams spending 2-3x in overall salaries. In fact, a Sports Illustrated article mentions; “From 2000 to 2006, (Oakland A’s) averaged 95 wins, captured four AL West titles and made five playoff appearances.”

Alas, all competitive advantage is fleeting. As Paul Depodesta left for greener pastures, and a few of Beane’s disciples took other league jobs, major league baseball teams figured out what made Moneyball work and effectively neutralized the Oakland A’s advantage.  With other teams adapting to Moneyball tactics, from 2007 and beyond, the Oakland A’s were a .500 team at best, and sometimes much worse.

And in the process of adopting Moneyball tactics, Billy Beane made plenty of enemies. The “old guard” definitely had a preferred way of defining baseball success and would often scoff at the “fat bodies” Beane employed, and laugh at the un-orthodox pitchers he would trot out in the ninth inning.  And in regard to the lackadaisical record of the Oakland A’s in recent years, one scout sneered; “So much for the genius … He doesn’t look so smart anymore, does he?”

Competing on analytics takes courage. In response to analyst and pundit criticism leveled at Beane for his Moneyball approach, author Michael Lewis says; “Beane had the nerve to seize upon ideas rejected, or at least not taken too seriously, by his fellow Club members, and put them into practice.”  For a while, Beane had the advantage. It might not have been pretty, but the Oakland A’s were effective at winning as many games as other prestigious teams, for half the cost.

And Beane ultimately may have the last laugh. Today, there are teams like the Boston Red Sox that take the best of Moneyball tactics and overlay them with powerful financial resources to take baseball competitiveness to a whole new level. Baseball has been changed forever from a “gut feel” business to one that’s analytically driven.

Fortunately for today’s analytical professionals, “competing on analytics” is taking hold as more companies understand they’re sitting on literal goldmines of structured and multi-structured data just begging for analysis. But analytics users be forewarned, there’s still plenty of “old guard” that will put up roadblocks, enlist subterfuge and even openly mock your new data driven approaches.

Michael Lewis notes that Beane didn’t invent sophisticated analytical analysis—he just had the courage to use it to create competitive advantage and shake up a traditionally stodgy industry.  There are plenty of industries ripe and ready for the business value that analytics will eventually unlock. Are you ready for the challenge?

Are Data Scientists the Next Masters of the Universe?

Back in the late 1970s, traders buying and selling mortgages were pushed aside for new masters of the universe—“quants” or individuals that used mathematics to slice and dice mortgages into debt tranches. And in the same way, today’s traditional Business Intelligence (BI) professionals must be looking over their collective shoulders as business and IT publications tout the emerging role of “data scientist”.

Before Lew Ranieri came on the scene, mortgages were a very staid business. Banks would loan money and keep assets on the books for up to thirty years (depending on how quickly the loan was paid back). Except for underwriting skills, there wasn’t much complexity to the mortgage business.

As a trader for Salomon Brothers, Lew Ranieri changed all that.  Ranieri’s insight was that mortgages could be bundled together and then sliced into different tranches of varied risk.  This slicing exercise was quite complex because of a buyer’s ability to prepay their loans early or refinance.  Michael Lewis, of Liar’s Poker fame writes; “Mortgages were acknowledged to be the most mathematically complex securities in the marketplace. The complexity arose entirely out of the option the homeowner has to prepay his loan…mortgages were about math.” 

Suddenly the very boring business of home loans became a very complex business challenge in how to slice the pie based on risk profiles and cash flows from interest and principal. Lewis writes; “Different investors place different prices on risk. Risk could be canned and sold like tomatoes.” And this mathematical complexity demanded a new skill set—quantitative analysis—to perform the necessary mathematical modeling to ensure investment banks remained profitable in this new business.

Pushed out by a new breed of mathematical whizz-kids, many former investment bankers and traders either retired or left for smaller financial firms. And the rise of the quants—or the new masters of the universe—was complete by the mid-1980s.

Is a similar shift happening in the field of Business Intelligence with the emerging “data scientist” role? The skill set of today’s data scientist is much more robust than one who solely performs BI or ETL application development.  With new sources and types of data (i.e. multi-structured), the data scientist must be able to develop new data driven products such as churn models, create recommendation algorithms, assist marketers with behavioral segmentation and targeting and more.

But that’s not all. Fellow SmartDataCollective contributor Daniel Tunkelang says the data scientist; “Also needs to possess creativity and strong communication skills. Creativity drives the process of hypothesis generation, i.e., picking the right problems to solve that will create value for users and drive business decisions.”  Tall order to find all these skill sets in one person, much less build an internal competency center with such talent.

Perhaps for the foreseeable future, there’s room for both traditional BI professionals and the new breed of data scientists, as today both are valuable contributors in the field of analytics. However, with data growth on a fast paced exponential curve, much less the complexity and velocity of multi-structured data, it’s easy to see how the mix of skill sets to succeed in the future will tilt more in favor of the data scientist role.

The mortgage bankers never saw Lew Ranieri coming. Regarding the rise of data scientists—should traditional BI professionals be worried?