Choose the right reference class for any prediction
Find the statistical base rate for the category your decision belongs to — not just the inspiring examples.
Key takeaways
- What it is: Find the statistical base rate for the category your decision belongs to — not just the inspiring examples.
- Why it works: When planning a project or decision, people naturally draw on the examples most vivid in their memory — which are disproportionately successes. Explicitly choosing a "reference class" — the set of all broadly similar undertakings — and looking up its historical outcome distribution forces a comparison to the whole population rather than to a curated sample of survivors.
- Evidence: Backed by observational / correlational evidence.
- Avoid: Choosing a flattering reference class ("projects like mine but successful ones") rather than the broadest accurate class, which reintroduces survivorship bias through the back door.
Why it works
When planning a project or decision, people naturally draw on the examples most vivid in their memory — which are disproportionately successes. Explicitly choosing a "reference class" — the set of all broadly similar undertakings — and looking up its historical outcome distribution forces a comparison to the whole population rather than to a curated sample of survivors.
How to do it
- 1Name the category your project or decision belongs to (e.g., "first-time restaurant opening", "software product launch", "career change to consulting").
- 2Look for the historical outcome distribution in that category: what percentage succeed, what’s the median outcome, what are the common failure modes?
- 3Use that distribution as your starting estimate before adjusting for factors specific to your case.
- 4Weight the base rate heavily unless you have concrete, specific reasons to expect your case is different.
What the evidence says
ObservationalReference class forecasting, formalized by Daniel Kahneman and Amos Tversky, shows that using the statistical distribution of a reference class consistently outperforms inside-view predictions. It was later developed into a planning tool by Bent Flyvbjerg in large infrastructure research.
Honest caveat: Reference class data is often difficult to obtain and the reference class itself is a judgment call. The method is most powerful in domains with existing historical data (construction, software projects) and less tractable where it does not exist.
- — Kahneman & Lovallo (1993), "Timid Choices and Bold Forecasts", Management Science — inside vs. outside view
Common mistake
Choosing a flattering reference class ("projects like mine but successful ones") rather than the broadest accurate class, which reintroduces survivorship bias through the back door.
IX Coach helps you identify and query the right reference class for goals you’re planning, grounding your probability estimates in distribution data rather than in the cases you happen to know about.
Practice this with IX Coach →