A Blog by Jonathan Low

 

Oct 12, 2017

Survivorship Bias: Why Success Can Rarely Be Validated Backwards

There is a tendency to believe that success can be replicated. Just follow the data, capture the patterns - hire the person or people who've done it - or something like it - and voila. 

But the reality, statistically and experientially, is that the past is rarely prologue for investors or managers. JL


Greg Satell reports in Digital Tonto:

Patterns can never be validated backward, only forward. So it doesn’t matter how many success stories we can database and then squeeze into graphs. The ugly little secret of business strategy is that you need to make your best guess in an uncertain context, manage resources wisely and see what works. Underlying every success story is the same promise: There are patterns to success and, by identifying them, we can map our own future. There is an element of truth to patterns, but they have little predictive value.
Every bold new business idea starts with a success story. Either it is a single organization or an aggregate sample that implemented a particular strategy and achieved outstanding results. That solid track record helps to convince others to adopt it, yet somehow the new management fad fails to deliver as promised.
The problem is often one of survivorship bias. While it’s fairly easy to find an examples of those who were successful with a particular strategy, the ones who tried it and failed are often overlooked. Other times, a post hoc fallacy is at work. Just because someone implemented a particular strategy doesn’t mean that’s what led to success.
The truth is that a strategy can never be validated backward, only forward. The past is a very imperfect indicator for the future because circumstances are constantly in flux. Technology, competition and customer preferences change all the time, so whenever anybody tells us that they have come up with a sure-fire way to succeed, we need to be skeptical.

Seeking Out Blue Oceans

In 2005 W. Chan Kim and Renée Mauborgne, both distinguished professors at INSEAD, published Blue Ocean Strategy. The idea was fairly simple: Why fight it out in a “red ocean” market filled with vicious competition, when you can seek out a “blue ocean” and create an entirely new market?  The book was an immediate hit, selling over 3.5 million copies.
The idea has a certain logic to it. Fighting for incremental market share gains is never fun and rarely leads to breakout profits. So, not surprisingly, in their study of 108 companies the authors found that “blue ocean” launches far outperformed the typical “red ocean” line extensions that are the norm in the corporate world.
Yet look a little closer and the concept quickly falls apart. 108 companies is a relatively small sample and tells us little about performance as a whole. Also, no competition also means no customers, so “blue ocean” markets are far more risky. What about the companies that went out of business betting on a new market that turned out not to exist?
So essentially what “blue ocean strategy” really comes down to is that riskier bets often have higher payoffs. Do we really need any fancy studies to tell us that?
Profiting From The Core
In 2004, just before Blue Ocean Strategy came out, Bain consultants Chris Zook and James Allen published their book, Profit from the Core. In a somewhat contrary view to Kim and Mauborgne,  they argue that firms that focus on their core business significantly outperform those who stray.
Like Blue Ocean Strategy, Profit from the Core was not based on pure conjecture. In fact, the Bain consultants boasted of a far more comprehensive study, encompassing 200 case studies, a database of 1,854 companies, 100 interviews of senior executives and an “extensive review” of existing literature. Wow! No stone, it seems, was left unturned.
The problem is that there are so many ways that you can define the “core,” — core markets, core capabilities and other “core” things — that the concept provides little actual guidance. The authors themselves even seem confused, praising Enron’s ability to “shift the core to drive into new horizons,” while panning Amazon’s move to sell products other than books.
So upon reviewing the literature of these distinguished scholars, we are left with the choice of sticking to the “core” or expanding into “blue oceans,” which is exactly where we’d be if we’d spent our time, money and effort drinking beer with friends rather than reading their books.

Platform Thinking

The newest hot idea is platforms and there have been a number of new books preaching the gospel, such as Platform Revolution, The Network Imperative and Matchmakers. Much like Blue Ocean Strategy and Profit from the Core, these boast significant studies, analyzing many companies with what seems like exceptional rigor. And who can deny the success of platform based businesses such as Facebook, Amazon, Uber and AirBnB?
Yet look a little closer and things don’t seem nearly as sanguine. AirBnB’s estimated revenues of $2.8 billion are impressive, but represent less than 1% of the $500 billion hospitality industry. Uber, faced with very low barriers to entry, has lost billions due to marketing expenses. That sure doesn’t seem so rosy.
In my book, Mapping Innovation, I profiled Elance, a platform business set up to connect firms and freelance contractors. It failed miserably. Later, it figured out how to create value over and above just making matches and built a great business. It eventually merged with competitor oDesk and now thrives as Upwork.
As I’ve noted before, platforms are immensely powerful because they help us to access ecosystems of talent, technology and information to create value. What value to create and for whom? Well, we still need to figure that out.

Overfitting The Future

Underlying every success story is the same promise: There are certain patterns to success and, by identifying them in the past, we can map out our own future. Like all myths, there is an element of truth to these patterns which makes them easy to believe, but they often have little predictive value.
Every enterprise develops core capabilities and thrives in core markets, but must build new capabilities and expand into new markets in order to grow. In much the same way, every organization today must develop proprietary capabilities and build direct customer relationships, while at the same time leveraging platforms to expand both.
That’s the problem with patterns. It is beyond our human capacity to swallow them whole, so we curate them to make them more comprehensible. In doing so, we eliminate important context and overfit selected elements of the past onto the future. That makes for a nice story, but fails to take into account the messy details of reality.
The truth is that patterns can never be validated backward, only forward. So it doesn’t matter how many success stories we can database and then squeeze into snazzy little graphs. The ugly little secret of business strategy is that you need to make your best guess in an uncertain context, manage resources wisely and see what works. There is no silver bullet.

1 comments:

Anonymous said...

This is a fantastic and quite useful article for me. Thank you so much for all of your hard work in giving such excellent information! buying property in Pakistan from UAE

Post a Comment