Conduct post-project debriefs to contribute to the class data
Record actual vs. forecast outcomes honestly — this builds the reference class that future forecasts depend on.
Key takeaways
- What it is: Record actual vs. forecast outcomes honestly — this builds the reference class that future forecasts depend on.
- Why it works: Reference class forecasting is only as good as the class data behind it. If practitioners never record honest outcomes, the class data available to future forecasters remains thin or systematically biased (projects that went badly are often not published). Disciplined post-project recording creates the base rate infrastructure that corrects the next round of planning.
- Evidence: Backed by observational / correlational evidence.
- Avoid: Conducting the debrief but not recording the outcome ratio in a searchable format — if it cannot be retrieved at the start of the next similar project, the learning is inaccessible.
Why it works
Reference class forecasting is only as good as the class data behind it. If practitioners never record honest outcomes, the class data available to future forecasters remains thin or systematically biased (projects that went badly are often not published). Disciplined post-project recording creates the base rate infrastructure that corrects the next round of planning.
How to do it
- 1After any significant project or prediction resolves, record: original estimate, final outcome, and ratio.
- 2Be specific about what drove the gap — was it a factor that could have been foreseen, or genuinely unpredictable?
- 3Store these in a personal or organizational forecast log that is referenced at the start of the next similar project.
- 4Resist the temptation to record only the projects that went well.
What the evidence says
ObservationalOrganizational learning research consistently finds that post-mortem/retrospective practices improve future performance when the data is honestly recorded and acted on. The infrastructure for reference class data in public projects was built precisely because individual organizations lacked systematic outcome records.
Honest caveat: Retrospective accounts are subject to hindsight bias and motivated reasoning; structured templates (pre-defined outcome metrics) reduce but do not eliminate this.
Common mistake
Conducting the debrief but not recording the outcome ratio in a searchable format — if it cannot be retrieved at the start of the next similar project, the learning is inaccessible.
IX Coach keeps a running log of your forecasts and their outcomes, making your personal base rate visible and searchable so each new plan benefits from the honest record of past ones.
Practice this with IX Coach →