Survivor’s Bias

3 min readSep 24, 2020

Survivorship Bias

Rahul Chhajed

Winston Churchill says, History is written by victors.

During World War 2, many of the American war planes were getting shot down and hence they wanted to strengthen them. From day one, the repair and maintenance team had kept a record of location of the bullet marks. Here is an image of the model of plane with red dots indicating bullet holes of all the planes that were brought into service stations.

Now, we all know the weight of the plane is very vital & too much armor to the plane and it would be too heavy to move swiftly. Therefore, the United States Air Force approached statistician Abraham Wald to come up with areas where extra Armour should be added. Before I present the conclusion of Dr. Wald, why don’t you pause for a moment and ask if you were an engineer working on this plane where you would armor your plane before continuing to read?

After a short while, Wald supplied his findings and recommendations: The USAF has thought that they need to armor the wings & tail of the airplanes. The recommendations Wald came up with surprised them. He instructed the Air Force to put the extra armor not where the bullet holes were, but where they weren’t — on the engines! His argument was, at least when you are shot on the wings and tail your aircrafts are making it back to the base where they can be repaired. But there was not a single plane that was shot on the Engine that made it back to the base. He had a point you see.

The Air Force followed his advice and the results were stunning. Immediately more planes started returning safely from combat saving the lives of countless pilots and crew members.

Wald correctly identified this as not so much a math problem but as a problem of survivorship bias, and once you understand this concept you start seeing it everywhere. Survivorship bias tells a lot of distorted and incomplete stories and looking for the missing bullet holes, as Wald had done, can save you from making bad decisions based on inaccurate and incomplete information.

What can we learn from survivorship bias:

Product managers ask us to design a certain product or feature replicating some competitors recent success. But for some reason, their users aren’t receptive. Why is that? Because most likely they paid attention to only success stories and forgot to understand backlash of their customers.

This bias is quite common in every field of life. More often than not, we have been introduced to such a bias. One of my friends who is a chain smoker brought my attention to Batuli Lamichhane, a Woman who lived 100+ years claiming that her long life is a result of her habit of smoking. Now, we do come across such stories many times in our life where we tend to focus only on success.

In both popular culture and business, we’re fascinated by successful people and successful brands. We want to know how they think and how they act. But few think to study those who tried — but never turned out to be successful.

I am not trying to sound negative or pull anyone away from following their own passion. Far from it. I am suggesting that we should always try to see the full picture. Being aware of survivorship bias and knowing how to avoid falling into it comes with massive upsides. On the surface, it helps you see through the incomplete information others provide intentionally. Inverse data sets are many times more causative and provide much more powerful clues to your original question. To summarize like Abraham Wald, who saved hundreds of soldier’s lives based on his simple observation, you take your blinders off. This could be all you need to do to save yourself frustration and save you your time, money and efforts.

References:

Survivor’s Bias

Written by Rahul Chhajed