Sanity-check against known extremes
Test your estimate against the clearly too-high and too-low bounds to calibrate your range.
Key takeaways
- What it is: Test your estimate against the clearly too-high and too-low bounds to calibrate your range.
- Why it works: Even when the precise estimate is unknown, the extremes are often knowable: the number of piano tuners in Chicago cannot be zero and cannot be ten million. Bounding the estimate from both sides constrains the plausible range and catches answers that drift outside the plausible region due to an error in the decomposition. The bounds provide a consistency check without requiring the answer itself.
- Evidence: Plausible mechanism, limited direct outcome data.
- Avoid: Setting the bounds symmetrically around your central estimate (e.g., ±50%) rather than thinking independently about what would need to be true for the extreme cases to hold.
Why it works
Even when the precise estimate is unknown, the extremes are often knowable: the number of piano tuners in Chicago cannot be zero and cannot be ten million. Bounding the estimate from both sides constrains the plausible range and catches answers that drift outside the plausible region due to an error in the decomposition. The bounds provide a consistency check without requiring the answer itself.
How to do it
- 1After your initial estimate, ask: what is the lowest it could plausibly be? What is the highest?
- 2Identify what would have to be true for the answer to be at each extreme.
- 3Check whether your estimate is inside the plausible range.
- 4If it is near an extreme, re-examine the decomposition step that is driving it there.
What the evidence says
MechanisticBounding techniques are a standard element in expert estimation and risk analysis. They reduce the impact of overconfidence and anchoring by forcing engagement with the tails of the distribution before settling on a central estimate.
Honest caveat: Sanity checks catch large errors but not small ones — an estimate can pass the extreme test while still being significantly off. They are a minimum check, not a verification.
Common mistake
Setting the bounds symmetrically around your central estimate (e.g., ±50%) rather than thinking independently about what would need to be true for the extreme cases to hold.
IX Coach asks you to state what would have to be true for your estimate to be wrong by 10x in each direction, making the bounding explicit rather than implicit.
Practice this with IX Coach →