Pre-commit to resolution criteria before making a prediction
Define exactly what counts as "I was right" before the outcome happens.
Key takeaways
- What it is: Define exactly what counts as "I was right" before the outcome happens.
- Why it works: Vague predictions survive almost any outcome — "I said things might get complicated" is consistent with disaster or success. This outcome-flexibility is a form of motivated reasoning that prevents calibration feedback from ever arriving. Pre-committing to specific, verifiable resolution criteria closes the escape hatch: the prediction either resolves correctly or it doesn’t, and you can’t post-hoc reconstruct what you "really" meant.
- Evidence: Established clinical or therapeutic practice.
- Avoid: Writing "I was right about the general direction" as retrospective resolution criteria after a disappointing outcome — this is hindsight reframing, not scoring.
Why it works
Vague predictions survive almost any outcome — "I said things might get complicated" is consistent with disaster or success. This outcome-flexibility is a form of motivated reasoning that prevents calibration feedback from ever arriving. Pre-committing to specific, verifiable resolution criteria closes the escape hatch: the prediction either resolves correctly or it doesn’t, and you can’t post-hoc reconstruct what you "really" meant.
How to do it
- 1When recording a prediction, write the resolution criteria in the same entry: "This is correct if X happens by Y date."
- 2Make criteria measurable: "more than 50% market share," "shipped by Q4," not "does reasonably well."
- 3Do not revise the criteria after the prediction is made — the revision is the measurement you’re trying to collect.
- 4Include a "null" condition: what would count as the prediction being indeterminate?
What the evidence says
Clinical practiceThe Good Judgment Project required all predictions to have explicit resolution criteria; this structural requirement is credited as a key reason its calibration measurements were meaningful. The practice is clinical/institutional rather than separately trialed as an intervention.
Honest caveat: Some real-world events resist clean binary resolution; the resolution criteria exercise is most powerful for near-term specific predictions and less clean for long-horizon, complex outcomes.
Common mistake
Writing "I was right about the general direction" as retrospective resolution criteria after a disappointing outcome — this is hindsight reframing, not scoring.
IX Coach locks in resolution criteria when you set a goal or forecast and uses them verbatim to evaluate the outcome, so each session’s record is a genuine calibration data point.
Practice this with IX Coach →