Practice calibration by tracking your confidence predictions
Your "80% confident" beliefs should come true about 80% of the time — check that they do.
Key takeaways
- What it is: Your "80% confident" beliefs should come true about 80% of the time — check that they do.
- Why it works: Calibration is the match between stated confidence and actual accuracy. Most people are overconfident: they assign high probabilities to beliefs that are true far less often than claimed. Deliberate calibration practice — predicting, then tracking outcomes — has been shown to meaningfully improve the accuracy of confidence estimates over time, because it provides the feedback loop that intuition normally lacks.
- Evidence: Backed by observational / correlational evidence.
- Avoid: Using vague language ("probably," "likely") instead of numbers, making it impossible to ever check whether your calibration has improved.
Why it works
Calibration is the match between stated confidence and actual accuracy. Most people are overconfident: they assign high probabilities to beliefs that are true far less often than claimed. Deliberate calibration practice — predicting, then tracking outcomes — has been shown to meaningfully improve the accuracy of confidence estimates over time, because it provides the feedback loop that intuition normally lacks.
How to do it
- 1Each week, write down five to ten predictions with explicit confidence levels (e.g., "75% confident the project ships on time").
- 2After the outcome is known, record whether you were right or wrong.
- 3Every month, compute your calibration for each confidence bucket: what fraction of your 70% predictions came true?
- 4Adjust your confidence levels based on the pattern — if your 80%s come true 60% of the time, you are overconfident.
What the evidence says
ObservationalCalibration training is one of the few documented interventions that reliably improves the accuracy of probabilistic predictions. Work by Phil Tetlock and colleagues on superforecasters found that explicit calibration practice was among the distinguishing features of the most accurate forecasters.
Honest caveat: Superforecaster research focuses on geopolitical and macroeconomic predictions; generalization to personal and professional decisions is plausible but less directly studied.
- — Tetlock & Gardner (2015), Superforecasting, Crown Publishers
Common mistake
Using vague language ("probably," "likely") instead of numbers, making it impossible to ever check whether your calibration has improved.
IX Coach tracks your confidence predictions over time and shows you a calibration curve — the single most useful feedback loop for improving probabilistic thinking.
Practice this with IX Coach →