State your prior probability before seeing the evidence
Before looking at any data, commit to a numerical estimate of how likely something is.
Key takeaways
- What it is: Before looking at any data, commit to a numerical estimate of how likely something is.
- Why it works: Naming a prior before seeing evidence does two things: it makes the subsequent update visible (you can see how far you moved), and it forces you to confront base rates — the background frequency of an event — rather than reasoning only from the case at hand. Research on base rate neglect shows that people systematically ignore background frequencies when case-specific information is available; a stated prior anchors them.
- Evidence: Backed by randomized trials / meta-analyses.
- Avoid: Setting a "prior" after glancing at the evidence, then believing you are reasoning from base rates when you are actually reasoning from the evidence twice.
Why it works
Naming a prior before seeing evidence does two things: it makes the subsequent update visible (you can see how far you moved), and it forces you to confront base rates — the background frequency of an event — rather than reasoning only from the case at hand. Research on base rate neglect shows that people systematically ignore background frequencies when case-specific information is available; a stated prior anchors them.
How to do it
- 1Before examining any new evidence, ask: "What is the base rate for this type of thing?"
- 2State a number: "I think there is roughly a 20% chance this is true before I look at any evidence."
- 3Write it down — a mental estimate evaporates under the pressure of incoming evidence.
- 4Only then look at the evidence, so you can measure how much it actually moves you.
What the evidence says
RCT / meta-analysisBase rate neglect is one of the most replicated findings in judgment research — people reliably underweight background frequencies when presented with individuating case information.
Honest caveat: The prior-setting habit as a practical countermeasure is widely endorsed in rationality training but has fewer direct experimental tests than the bias it targets.
- — Kahneman & Tversky (1973), on the psychology of prediction, Psychological Review
Common mistake
Setting a "prior" after glancing at the evidence, then believing you are reasoning from base rates when you are actually reasoning from the evidence twice.
IX Coach asks you to commit to a probability estimate before presenting any supporting information, creating a genuine before-and-after comparison rather than letting evidence absorb your prior.
Practice this with IX Coach →