How To Be a Good Forecaster
The following ten commandments are lifted from Superforecasting by Tetlock.
- Triage: Focus on questions where hard work is likely to pay off.
- Break seemingly intractable problems into tractable sub-problems: Perform the fermi estimate. Expose and examine assumptions.
- Strike the right balance between inside and outside views. Nothing is 100% unique. See Outside/Inside View.
- Strike the right balance between over and under-reacting to evidence. Bayesian belief updating pays off in the long term. This requires skillful processing of news flows.
- Look for clashing causal forces at work in each problem. List all the arguments for and against a point, that are at least worth acknowledging.
- Strive to distinguish as many degrees of doubt as the problem permits, but no more. Think in more granular ways about uncertainty (going fully probabilistic helps).
- Strike the right balance between under and overconfidence, between prudence and decisiveness. Long-term accuracy is dependent on both calibration and resolution.
- Look for errors behind your mistakes, but beware of hindsight bias. Conduct postmortems on forecasting failures and successes.
- Bring out the best in others and let others bring out the best in you. Forecasting as a team can bring out the wisdom of a crowd. Let people bring in different perspectives, or provide constructive questioning.
- Master the error-balancing bicycle. Getting good at forecasting requires deliberate practice, and learning from feedback about whether you are succeeding.