Notice which data you selected — and which you ignored
Your conclusions are built on a sample of the available data — ask what the sample excluded.
Key takeaways
- What it is: Your conclusions are built on a sample of the available data — ask what the sample excluded.
- Why it works: Attention is finite, and selection is always happening, usually in the direction of information that confirms existing beliefs. The data you selected feels like "the data" — the rest does not feel excluded, it feels absent. Deliberately searching for the data you did not attend to is the countermeasure to selection bias operating at the bottom of the ladder.
- Evidence: Backed by observational / correlational evidence.
- Avoid: Selecting data to support the opposite conclusion as the "corrective" — the goal is to find data you genuinely weighted too low, not to perform the opposite bias.
Why it works
Attention is finite, and selection is always happening, usually in the direction of information that confirms existing beliefs. The data you selected feels like "the data" — the rest does not feel excluded, it feels absent. Deliberately searching for the data you did not attend to is the countermeasure to selection bias operating at the bottom of the ladder.
How to do it
- 1After reaching a conclusion, ask: what did I notice, and what was I not attending to?
- 2Seek out data that a person who reached the opposite conclusion would have selected.
- 3Ask someone who knows the same situation whether they weighted the same events you did.
- 4Treat any data point you feel the urge to dismiss as worth examining more carefully.
What the evidence says
ObservationalSelective attention and confirmation bias are among the most replicated findings in cognitive psychology. People attend to and remember information consistent with existing beliefs more reliably than inconsistent information.
Honest caveat: Awareness of confirmation bias provides only partial protection; the selection process operates partly below consciousness and is not fully correctable by attention alone.
- — Nickerson (1998), "Confirmation Bias: A Ubiquitous Phenomenon in Many Guises", Review of General Psychology
Common mistake
Selecting data to support the opposite conclusion as the "corrective" — the goal is to find data you genuinely weighted too low, not to perform the opposite bias.
IX Coach asks what data you did not weight heavily when it hears a confident conclusion, surfacing the selection bias before the reasoning continues.
Practice this with IX Coach →