Developed by Sarah Lichtenstein and Baruch Fischhoff (Training for Calibration, 1980) with Lawrence D. Phillips (Calibration of Probabilities, 1982), brought to business by Douglas W. Hubbard (How to Measure Anything, 2007), and carried into forecasting by Philip E. Tetlock and Dan Gardner (Superforecasting, 2015).
Drills people on questions with known answers, scoring how often their 90 percent ranges contain the truth, until stated confidence matches the actual hit rate.
When to reach for it
Before people are asked for probability estimates or ranges, or when a team's estimates keep proving too confident.
How it runs
- Give 20 to 40 questions with numeric answers.
- Have each person state a 90 percent range for each.
- Score how many ranges caught the true answer.
- Teach the equivalent-bet test for widening a range.
- Repeat with fresh questions until hit rates approach the stated confidence.
Where it sits in SOLVE eX
- Any step: Tools that serve every step: habits of thought and checks on the thinking itself.
Go to the source
- Training for Calibration, Organizational Behavior and Human Performance 26, 1980
- Calibration of Probabilities: The State of the Art to 1980, Judgment under Uncertainty: Heuristics and Biases, 1982
- How to Measure Anything: Finding the Value of Intangibles in Business, 2007
- Superforecasting: The Art and Science of Prediction, 2015