Case Studies

This post is contributed by Kathleen Maley, VP, Analytics Executive and Member of the IIA Expert NetworkFor Billy Beane, in the movie “Moneyball”, becoming data-driven was personal. As a young baseball player, he had all the right tools to make it to the Major Leagues. He could catch, run fast, throw hard, and hit the ball often and far. Talent scouts told him he could be an all-star and convinced him to give up a scholarship to Stanford University. Their experience told him he couldn’t miss. But they were wrong, and Beane never came close to justifying his first-round draft pick.Later, as general manager of the Oakland A’s, faced with a shoestring budget and richer teams luring some of his star players away with big money, Beane feared a conventional staffing approach wouldn’t be enough to replace lost talent. Then he met a geeky, Ivy League economics grad who had little experience evaluating players’ physical abilities, but had embraced a different way of evaluating performance data. In the movie, Beane made Peter Brand (whose name in real life is Paul DePodesta) assistant general manager and made sure he was in the room when trades and other personnel moves were discussed. They replaced departed stars with players who were cheaper, but that was because nobody else -- including A’s fans -- wanted them.The A’s started slowly in 2002 -- so slowly that Beane, Brand and just about everyone else doubted the new approach -- but then they caught fire and surprised everyone by setting an American League record for consecutive wins and making the playoffs. The obvious takeaway from this story is the one with which we are all familiar: Data can play a vital role in engineering a desired outcome. But Beane’s experience, as portrayed in “Moneyball”, reveals other critical takeaways as well.

Surprising insights = competitive advantages

Brand explains in his first encounter with Beane that people running baseball teams mistakenly think in terms of buying players. “Your goal shouldn’t be to buy players. Your goal should be to buy wins. And in order to buy wins, you need to buy runs.”As someone trained in the study of data, Brand instinctively asked: What is the desired outcome, and what set of performance statistics will predict the desired outcome? That helped Beane see opportunities that others in baseball couldn’t.The value in approaching a problem from an analytical perspective isn’t unique to Moneyball. Very early in my career, I helped a doctor studying the effects of long-term dialysis on arterial elasticity. I have no medical training, and I took my last biology class in high school. In our first meeting, we discussed the outcome that mattered and the data she had collected. She shared the hypothesis she was testing and told me which data mattered and which didn’t. She told me to focus on the elasticity of small arteries and to not waste time looking at large arteries, because conventional wisdom was clear that large arteries are virtually inelastic.But I’m a data scientist, and my job is to study data. I was skeptical of her assertion that certain data didn’t matter and that large arteries are inelastic. I was able to run my analysis twice for almost zero extra cost, so I ran it on both the large and small arteries. Her hypothesis about the small arteries held. She ended up writing her paper, however, on the increased elasticity of the large arteries that my analysis had discovered.The doctor had much more knowledge of the human circulatory system than I did, but in this instance, her knowledge of conventional wisdom discouraged her from even attempting to find out if it was really true. My willingness to test assumed truths gave her better analysis than she would have gotten if I had merely fulfilled her request, and, to her credit, she switched gears and wrote a paper that changed -- in a small but significant way -- our understanding of the human body.If you’re lucky enough to have input from a good data analyst, define the desired outcome and then let your analyst study its relationship to upstream information. She will see things in the data that will surprise you, and that’s where the competitive edge resides.

Key Takeaway: To gain a competitive advantage, you have to use your data in ways others aren’t. Define your broader business goal and leave the data decisions to your analyst.

Augment, collaborate, innovate, transform

When Beane hired Brand, A’s scouts could see only that he wasn’t an experienced talent scout like them. Whether through hubris or a perceived affront to their expertise, they dismissed him.Although typically less overt, this reaction is still common within many organizations during the early stages of digital transformation. Some business leaders might feel threatened or challenged. Some simply feel that bringing a “data person” to the table is more work than it’s worth.And, just as problematic, in many cases it’s the analyst who thinks about data in isolation instead of within the context of business objectives. Whatever the case, the resulting lack of collaboration between the business experts and the analytics experts comes at a bottom-line cost.When each party approaches the partnership as consultative, however, a more productive dynamic develops. The skills, knowledge, and experience of each are augmented by those of the other, creating a whole that is greater than the sum of its parts.

Key Takeaway: Analytics partners should operate as strategic thought partners, not just order takers serving up requested data.

If it feels risky, you’re doing it right

Neither Beane nor Brand were sure a data-driven approach would work, because neither of them had done it before. Brand had theory, Beane had baseball experience and desperation, and together they took a risk. Most business leaders today are familiar with data and analytics, but lingering doubt still shows itself in staffing decisions. While there is recognition that analytics matters as a distinct job responsibility, that often translates into hiring someone familiar with the business to run a data and analytics team. They want someone who can help them find new ways to use analytics, but they want that someone to be just like them.It’s understandable, especially considering the all-too-common language barrier that persists between many analysts and the rest of the organization. To an analyst, discussing the “coefficient of determination” is like chugging Red Bull, but to a manager, it’s like sipping Sleepytime tea. However, hiring a manager instead of an analyst also doesn’t introduce the complementary skill set that analytics experts possess.That’s not to say that traditional experience in a business doesn’t matter. It matters a lot. (In real life, for example, the A’s got valuable contributions from many players identified by their traditional scouts.) But almost without exception, a trained analyst with strong consulting skills will be able to learn what matters most to the business far more quickly than a business leader can be formally trained in applied analytics and data science. An analytics leader with the ability to relate to multiple business leaders in an organization -- and fully manage and unlock the capabilities of an analytics team -- can help a business in ways far beyond what most imagine.

Key Takeaway: An analytics leader with a consultative mindset will be able to extract more value from your data and analytics team than a business leader who lacks analytical training.

Material improvements, but no silver bullet

Ultimately, the Moneyball approach did not yield a World Series Championship for the Oakland A’s. It helped Beane and his coaches evaluate players more clearly and helped the team stay competitive, but it wasn’t a cure for a small budget. The wealthier Boston Red Sox adopted a similar approach and, a few years later, won a World Series for the first time in 86 years. The A’s and Red Sox were both better with a data-driven approach than a purely traditional one, but the Red Sox also had other resources necessary to win baseball’s biggest prize. These days, every Major League team relies on data analysts to help evaluate players.There is a widely held misconception that if an analytics approach is effective, then it can operate independently to produce miraculous results for the business. This is generally not the case and businesses shouldn’t expect an analytical approach -- especially one that operates without continual input from frontline managers -- to produce miracles on its own. By adopting the mindset that “there is no such thing as an analytics initiative, only a business initiative”, analytics finds its proper place as an enabler of business outcomes. When fully integrated into business functions through an effective engagement model and strategic framework, analytics can be used to help business leaders aim higher and get there faster.

Key Takeaway: Analytics teams aren’t a cure-all that magically produce big profits; but they do wield a very powerful tool that can help an organization see itself more clearly and operate more effectively.

About Kathleen Maley

An experienced analytics executive and P&L owner, Kathleen Maley specializes in making data and analytics work for organizations that are challenged to bridge the divide between data and business value. By combining strategic vision, technical expertise and highly developed emotional intelligence, she effectively guides the evolution of data analysts and business leaders into thought partners and data-enabled decision makers. Kathleen is a published writer and frequent speaker and holds degrees in Mathematics (BA) and Applied Statistics (MA).