Jethro's Braindump

How To Be a Good Forecaster

The following ten commandments are lifted from Superforecasting by Tetlock.

  1. Triage: Focus on questions where hard work is likely to pay off.
  2. Break seemingly intractable problems into tractable sub-problems: Perform the fermi estimate. Expose and examine assumptions.
  3. Strike the right balance between inside and outside views. Nothing is 100% unique. See Outside/Inside View.
  4. Strike the right balance between over and under-reacting to evidence. Bayesian belief updating pays off in the long term. This requires skillful processing of news flows.
  5. Look for clashing causal forces at work in each problem. List all the arguments for and against a point, that are at least worth acknowledging.
  6. Strive to distinguish as many degrees of doubt as the problem permits, but no more. Think in more granular ways about uncertainty (going fully probabilistic helps).
  7. Strike the right balance between under and overconfidence, between prudence and decisiveness. Long-term accuracy is dependent on both calibration and resolution.
  8. Look for errors behind your mistakes, but beware of hindsight bias. Conduct postmortems on forecasting failures and successes.
  9. Bring out the best in others and let others bring out the best in you. Forecasting as a team can bring out the wisdom of a crowd. Let people bring in different perspectives, or provide constructive questioning.
  10. Master the error-balancing bicycle. Getting good at forecasting requires deliberate practice, and learning from feedback about whether you are succeeding.