“The fox knows many things, but the hedgehog knows one big thing.” ~ Archilochus
Philip Tetlock — Why Foxes Are Better Forecasters Than Hedgehogs floratv
Study, 28,000 predictions, 284 experts, most with advanced degrees, from the government, journalists, academia, and others.
Found forecasting accuracy is determined by how one thinks, rather than what one thinks; meaning, traditional divisions of opinion – libertarian versus Marxist – carry little weight in prediction accuracy.
One should be able to consider two sides of the argument, think in terms of probabilities rather than certainties, and be able to hold conflicting thoughts.
Hedgehogs: "relate everything to a single central vision. …in terms of which all that they say has significance." They over simplify, don’t use diverse data sources.
Foxes: "pursue many ends, often unrelated and even contradictory….entertain ideas that are centrifugal rather than centripetal;…..without seeking to fit them into, or exclude them from, any one all-embracing inner vision."
The closer to the blue line the better: foxes are the pink line, while the hedgehogs forecasted only in their domain expertise and did the worst.
One should approach things from a deductive standpoint – start with first principles, accumulate many different facts into those first principles, often contrary to each other. Antithetical to science, which attempts to deduce big theories from limited data.
Those that predicted the Soviet Union’s collapse due to the nature of communism, were proven right, but many had been making that prediction since the ’60s and ’70s.
Aggregating hedgehogs does much better than individual hedgehogs; much less incremental improvement occurred from aggregating opinion of foxes.
When a forecaster uses however and buts, it shows they are correctly layering contradictions. The problem is this type of presentation bores audience, so the predictions of foxes don’t make the light of day.
Only a very few outperformed models, especially more advanced models.
Some evidence knowing that your opinions are being monitored improves prediction.
Note: In this presentation, Tetlock focused much less on model predictions, but in his book and elsewhere, he found aggregating many different models to outperform almost both hedgehogs and foxes.
Popularity: 8% [?]