How does "Update beliefs frequently and in small increments" work?

Superforecasters update more often and in smaller increments than average forecasters. The mechanism is Bayesian: each new piece of evidence has a diagnostic weight, and updating in proportion to that weight produces calibration. The two failure modes are under-updating (anchoring on the original estimate) and over-updating (overweighting recent salient events). Small, frequent updates resist both by keeping the estimate close to current evidence.

How does "Actively seek disconfirming evidence" work?

Confirmation bias causes people to weight confirming evidence more than disconfirming evidence, producing overconfidence. Superforecasters counteract this by making disconfirmation search an explicit habit: they ask "what would prove me wrong?" before seeking supporting evidence. This does not eliminate the bias but changes the evidence set the estimate is drawn from.

How does "Decompose complex questions into sub-questions" work?

Forecasting a complex outcome directly invites an availability-based gut estimate, which is sensitive to framing and salient recent events. Decomposing into sub-questions forces the analysis to address each component independently, catches missed dependencies, and makes the estimate auditable. Aggregating well-calibrated sub-estimates produces more accurate total estimates than directly estimating the whole.

How does "Treat forecasting accuracy as a skill that improves with practice" work?

Superforecasters treat their scoring results as diagnostics, not verdicts. When they are miscalibrated in a domain, they investigate why — was the reference class wrong, was the evidence weighting off, was there a structural dependency they missed? This feedback orientation is the same mechanism that distinguishes deliberate practice from mere experience. Without it, repetition produces confidence without accuracy.

How does "Use structured team forecasting to aggregate diverse views" work?

Group discussion tends to amplify the views of the most confident or senior member and suppress minority information. Aggregating independent estimates before discussion preserves information that discussion would wash out. The wisdom of crowds effect depends on independence; the mechanism breaks down the moment estimates are made in the presence of others’ views.

Philip Tetlock — Superforecasting

How do superforecasters make more accurate predictions than experts?

The habits, mindset, and calibration practices of the world’s most accurate forecasters

Short answer

Philip Tetlock’s forecasting tournament research found that a subset of ordinary people — "superforecasters" — consistently outperform domain experts and intelligence analysts at probabilistic prediction. They share measurable cognitive and behavioral habits: they think in probabilities, update frequently on evidence, and actively seek disconfirming information. These habits are learnable.

In the Good Judgment Project, Tetlock and colleagues ran the largest forecasting tournament in history and found something unexpected: a small group of non-expert volunteers outperformed CIA analysts with access to classified intelligence. The superforecasters were not smarter overall — they deployed a specific set of cognitive practices that most people can adopt. Below are those practices, with the mechanism behind each and an honest account of the evidence.

The practices (6)

Why it works

Words like "likely" are systematically ambiguous — one person’s "likely" is another’s 55% and another’s 85%. Numerical probabilities force specificity, make predictions scorable, and create a feedback loop. Without numbers, there is no signal for whether you are calibrated. The discipline of naming a number also forces genuine uncertainty accounting — you cannot say "basically certain" when you have to commit to 95%.

How to do it

1When making a prediction, state it as a percentage: "I think there’s a 65% chance this ships by Q3."
2Record the number alongside the prediction so you can score it when the outcome resolves.
3When someone gives you a verbal probability, ask: "What number would you put on that?"
4Revisit old predictions with their numbers quarterly to compute your calibration.

Evidence

Observational

Tetlock’s forecasting research found that superforecasters habitually used numerical probabilities and were significantly better calibrated than experts who used verbal descriptions. The practice of using fine-grained probabilities (not just 50/70/90) was associated with higher accuracy in the tournament.

Honest caveat: Correlation between probability-use and accuracy in tournament data; calibration training with feedback is needed for the habit to produce real improvement.

— Tetlock & Gardner (2015), Superforecasting
— Tetlock (2005), Expert Political Judgment

Common mistake: Using round numbers (50%, 70%, 90%) as a default rather than genuinely calibrating — "I think 50/50" is often a hedge, not an estimate.

Go deeper on this practice →

Practice this with IX Coach

Reading about a practice changes nothing on its own. IX Coach turns these into a guided, adaptive routine — discerning where you are in real time and walking the practice with you, session after session.

Start with IX Coach →