Update the map from real feedback
When reality contradicts your model, revise the model — not your description of reality.
Key takeaways
- What it is: When reality contradicts your model, revise the model — not your description of reality.
- Why it works: Cognitive dissonance between a confident model and inconvenient evidence can be resolved either by updating the model or by reinterpreting the evidence to fit. The path of least resistance is usually the latter — dismissing the anomaly as an outlier, attributing it to bad luck, or mislabeling the evidence. Explicitly committing to update the model rather than explain away the evidence is a deliberate override of this default.
- Evidence: Backed by observational / correlational evidence.
- Avoid: Revising the framing of the outcome ("that doesn’t count because the context was unusual") rather than the model that generated the wrong prediction.
Why it works
Cognitive dissonance between a confident model and inconvenient evidence can be resolved either by updating the model or by reinterpreting the evidence to fit. The path of least resistance is usually the latter — dismissing the anomaly as an outlier, attributing it to bad luck, or mislabeling the evidence. Explicitly committing to update the model rather than explain away the evidence is a deliberate override of this default.
How to do it
- 1Before an action, write down what outcome your model predicts.
- 2After the outcome, compare prediction to reality without softening the discrepancy.
- 3Identify which specific assumption in your model the discrepancy is evidence against.
- 4Revise that assumption, however uncomfortable, before the next decision.
What the evidence says
ObservationalProspective prediction-tracking is one of the most reliable ways to identify model errors, used systematically in superforecasting research. Forecasters who track and score their predictions calibrate substantially better than those who do not.
Honest caveat: Prediction tracking requires honest pre-commitment; people are prone to "I knew it all along" reinterpretation without the external record.
- — Tetlock & Gardner (2015), Superforecasting — tracking records vs unaided retrospective judgment
Common mistake
Revising the framing of the outcome ("that doesn’t count because the context was unusual") rather than the model that generated the wrong prediction.
IX Coach tracks what you expected and what happened, and surfaces the gap as a prompt to examine your model rather than explain it away.
Practice this with IX Coach →