Practice probabilistic calibration by tracking your predictions
Assign explicit probability estimates to your predictions and track whether they come true at the right rate.
Key takeaways
- What it is: Assign explicit probability estimates to your predictions and track whether they come true at the right rate.
- Why it works: Calibration — the match between stated confidence and actual accuracy — is a learnable skill. People who track their predictions over time and observe how often 70% confident claims actually come true develop feedback loops that correct systematic over- or under-confidence. Without this feedback, the confident feeling generated by vivid information persists uncorrected.
- Evidence: Backed by observational / correlational evidence.
- Avoid: Assigning vague probabilities ("pretty likely") rather than specific numbers, which prevents meaningful calibration checks because the prediction is unfalsifiable.
Why it works
Calibration — the match between stated confidence and actual accuracy — is a learnable skill. People who track their predictions over time and observe how often 70% confident claims actually come true develop feedback loops that correct systematic over- or under-confidence. Without this feedback, the confident feeling generated by vivid information persists uncorrected.
How to do it
- 1For any prediction you make, assign a percentage probability: "I think there’s a 70% chance X happens."
- 2Record the prediction and the outcome.
- 3After 20–30 predictions, check your calibration: are your 70% predictions coming true about 70% of the time?
What the evidence says
ObservationalCalibration research (including work on superforecasters by Tetlock) shows that tracking predictions with explicit probabilities significantly improves accuracy over time through feedback-driven error correction.
- — Tetlock & Gardner (2015), Superforecasting — calibration training and improvement across thousands of predictions
Common mistake
Assigning vague probabilities ("pretty likely") rather than specific numbers, which prevents meaningful calibration checks because the prediction is unfalsifiable.
IX Coach tracks predictions you make during coaching sessions and resurfaces them for calibration review — turning your confidence statements into data rather than just feelings.
Practice this with IX Coach →