How does "Actively seek out the failures you aren’t seeing" work?

The defining feature of survivorship bias is that the missing data is invisible by design — failed businesses close, silent failures don’t publish, and losers generally don’t give keynotes. Reversing this requires deliberate effort to seek out the non-surviving cases: people who quit, companies that closed, strategies that didn’t work. The act of looking is itself the corrective, because the failures are there — they’re just not findable without a specific intention to find them.

How does "Choose the right reference class for any prediction" work?

When planning a project or decision, people naturally draw on the examples most vivid in their memory — which are disproportionately successes. Explicitly choosing a "reference class" — the set of all broadly similar undertakings — and looking up its historical outcome distribution forces a comparison to the whole population rather than to a curated sample of survivors.

How does "Audit whether your information sources systematically favor survivors" work?

Survivorship bias is often structural: the channels through which knowledge flows — bestseller lists, business journalism, social media success stories, speaker circuits — are themselves selection mechanisms that favor outlier successes. The information environment doesn’t represent the distribution; it represents the tail of it. Auditing your sources for this filter helps you weight what you read against the knowledge that the source was never designed to show you failures.

How does "Build a personal library of failure post-mortems" work?

Success cases teach you how one path to success looked — a thin signal given the survival filter. Failure post-mortems, by contrast, carry dense information about mechanisms: the specific decisions, assumptions, and dynamics that reliably lead to bad outcomes. Because the survivorship filter makes failures invisible by default, deliberately curating them is the counterweight that gives you access to the full distribution of what happens.

How does "Steelman the strategy opposite to the successful one" work?

When survivorship bias has already selected your examples, reasoning within the frame of those examples compounds the bias. Requiring yourself to build a genuine case for the opposite strategy — not a strawman, but the strongest possible version — forces you out of the frame and confronts the selection filter directly. If you can’t construct a plausible case for the opposite, the lesson may be real; if you can, you’ve found where survivorship bias was doing the work.

Abraham Wald

What is survivorship bias and how does it distort decisions?

The invisible graveyard of failures — and how to reason from the full distribution

Short answer

Survivorship bias is the error of drawing conclusions only from the cases that made it through a filter — winners, survivors, visible successes — while the failures that never appear are silently excluded. The clearest historical example is Abraham Wald’s WWII aircraft study: the military wanted to armor the bullet holes they saw on returning planes; Wald showed they should armor where they saw no damage, because planes hit there didn’t return.

You learn from what you see. Survivorship bias makes the sample you see systematically unrepresentative: the failures, the dead companies, the strategies that wrecked people are invisible, so the pattern you perceive is shaped by the selection filter, not by the actual distribution. The result is overconfidence in strategies with high variance and high casualty rates. Below are the practices for catching and correcting this pattern.

The practices (6)

Why it works

Any strategy, regardless of how random, will produce vivid successes among a large enough pool of attempts. The success itself tells you very little about the quality of the strategy — you need the denominator (how many tried) to compute the actual probability. Without the failure rate, your brain is pattern-matching on selected survivors, not on representative outcomes.

How to do it

1When you encounter any success story or case study, ask: "How many people or businesses tried this same approach?"
2Try to find the failure rate or base rate for that category.
3If the failure rate is unavailable, hold the lesson with explicit uncertainty rather than treating it as a reliable lesson.
4In your own decisions, log attempts and failures alongside successes so you have your own base rate.

Evidence

Observational

The failure-rate question is the direct debiasing move for survivorship bias. Base-rate neglect — ignoring background frequencies — is one of the most robust findings in the judgment-under-uncertainty literature. Forcing explicit consideration of base rates significantly improves probabilistic reasoning.

Honest caveat: Knowing the base rate improves reasoning in tasks where the base rate is provided. In real-world settings, the failure-rate data is often genuinely unavailable, and even when available, people anchor to the vivid case rather than the statistic.

— Kahneman & Tversky (1973), "On the Psychology of Prediction", Psychological Review — base-rate neglect

Common mistake: Assuming that if the success story exists the strategy is worth emulating, without asking how representative that story is. Celebrity biographies and business case studies are almost entirely drawn from survivors.

Go deeper on this practice →

Practice this with IX Coach

Reading about a practice changes nothing on its own. IX Coach turns these into a guided, adaptive routine — discerning where you are in real time and walking the practice with you, session after session.

Start with IX Coach →