How understanding what’s missing can transform your strategy.
In the world of marketing, success stories are everywhere. We see viral campaigns, skyrocketing product launches, and brands gaining cult-like followings seemingly overnight. But how often do we stop to think about the failures and near-misses that don’t make the headlines? Enter the concept of survivorship bias, a fascinating phenomenon that challenges the way we interpret results.
What is Survivorship Bias?
Survivorship bias occurs when we focus only on the “survivors”—the successful outcomes—while ignoring the many failures that fell along the way. It’s like studying a room full of lottery winners and trying to learn their “secrets” to success, ignoring the thousands of people who didn’t win despite playing the same game.
In marketing, this bias often leads us to draw conclusions from the most visible successes without considering the broader context. This leads to flawed decisions and strategies that overlook key lessons.
Diagram in which red dots stand for places where surviving planes were shot. This only tells you where planes can get shot and still come back to base. Survivorship bias: your only information is what has survived.
A Historical Perspective: Cicero and Diagoras
Thousands of years ago, the Roman philosopher and politician, Cicero, told the story of an atheist named Diagoras, who had been vocal in his disbelief about the gods. Those trying to convince Diagoras of the existence of higher powers showed him a series of painted tablets that portrayed a group of sailors who had prayed during a vicious storm and then survived the shipwreck.
Diagoras looked at the paintings and replied, “I see those who were saved, but where are those painted who prayed and drowned?”
Diagoras, through Cicero’s retelling, highlighted an all-too-common flaw in human reasoning and decision-making: We ground ourselves in the seen and systematically ignore the unseen. We admire the “winners” and study their success stories in hopes of determining the common traits that led to their success, but we fail to recognize the many who took similar risks and did not succeed.
This insight from Diagoras still holds today. We long for simplicity, but it misses critical underlying realities. When we look only at the successful outcomes, we miss out on learning from those that didn’t survive.
The Statistician Who Saved Lives: Abraham Wald’s Insight
To better understand how survivors can distort our perspective, let’s take a look at the story of Abraham Wald, a statistician whose work during World War II provides an eye-opening lesson on survivorship bias.
Wald was part of a research group tasked with reducing casualties in the U.S. Air Force. The military was studying the bullet holes in planes that made it back to base. They noticed that the wings and tail were often damaged by bullets, while the engines seemed to remain intact. The obvious conclusion was to reinforce the wings and tail with more armor. But Wald had a different insight.
He recognized that the planes that returned were survivors, and the ones that didn’t return likely had damage to the engines. His recommendation? Arm the engines, not the wings or tail. This simple, yet profound insight saved countless lives, demonstrating how looking only at survivors can lead us to flawed conclusions.
How Survivorship Bias Affects Marketing
When survivorship bias seeps into our decision-making, it can create unrealistic expectations and blind us to valuable lessons. Here’s how:
- Overvaluing Viral Campaigns Marketers often study viral campaigns to replicate their success. However, for every viral hit, countless similar campaigns fail to gain traction. By focusing only on the winners, we miss crucial insights about what didn’t work and why.
- Ignoring Silent Failures When a marketing strategy doesn’t deliver results, it’s often quietly discarded and forgotten. Yet, these “failures” may hold the key to understanding audience preferences, market trends, or internal inefficiencies.
- Creating Unrealistic Benchmarks Survivorship bias can lead to unrealistic expectations for performance. If we measure ourselves against only the most successful campaigns, we risk undervaluing steady, sustainable growth.
How to Avoid Survivorship Bias in Marketing
- Analyze Both Successes and Failures Don’t just focus on the campaigns that worked—look at the ones that didn’t. Understanding why something failed can provide insights that are just as valuable as understanding why something succeeded.
- Ask the Right Questions When analyzing a successful campaign, dig deeper:
- What unique circumstances contributed to its success?
- Could these conditions be replicated?
- What might have caused it to fail?
- Diversify Your Learning Sources Instead of relying solely on case studies of top-performing campaigns, study broader trends and collect data from a range of outcomes.
- Test, Measure, Adjust Avoid placing all your bets on replicating a past success. Embrace testing and iteration to create strategies that are adaptable and data-driven.
The Takeaway
Survivorship bias reminds us that marketing is about more than just results—it’s about understanding the entire journey. Success and failure are two sides of the same coin, and both offer invaluable lessons if we’re willing to look beyond the surface.
By acknowledging and addressing survivorship bias, marketers can make smarter decisions, set more realistic goals, and create campaigns that are built for long-term success.
Have you encountered survivorship bias in your own work? Share your thoughts below—I’d love to hear how you’ve tackled this challenge!
Extra Context:
If you’re still curious about how survivorship bias operates in various industries, consider the story of Abraham Wald during World War II. Wald’s insight into the military’s study of bullet holes in returning planes is an excellent example of why we must take a step back and view the whole picture, not just the visible successes. His research helped the U.S. Air Force realize that focusing only on the planes that made it back could lead to dangerous miscalculations about what parts needed to be reinforced. It wasn’t the areas where the planes were hit that needed more armor—it was the engines, the very component that would cause a plane to crash if damaged.
The same kind of thinking applies in marketing. We often focus on successful campaigns and brands, but without considering the factors behind both the failures and successes, we miss important lessons that could guide us to more strategic decisions
Sources
- McDreeamie’s Musings
Survivorship Bias: How Lessons from World War Two Affect Clinical Research Today
This blog post explores how survivorship bias, demonstrated through Abraham Wald’s insights during World War II, impacts fields like clinical research. It delves into real-world applications of recognizing unseen data and avoiding flawed conclusions. - Cameron Moll
Abraham Wald and the Red Bullet Holes Origin Story
This article recounts the story of Abraham Wald and his contributions to mitigating survivorship bias during World War II. It highlights the broader implications of understanding this cognitive bias. - Ross Griffin
Survivorship Bias
This resource provides an in-depth analysis of survivorship bias, its origins, and its influence across various domains. It offers insights into recognizing and addressing this bias in decision-making processes.
