Philip Tetlock, The Expert who Models Experts

July 8, 2011


Tetlock signed up nearly 300 academics, economists, policymakers and journalists and mapped more than 82,000 forecasts against real-world outcomes.

Most failed over random forecasts, but Tetlock’s research found one kind of expert turns out consistently more accurate forecasts than others.

  • “The better forecasters were eclectic thinkers who were willing to update their beliefs when faced with contrary evidence, were doubtful of grand schemes and were rather modest about their predictive ability.”
  • They often qualify their arguments with “however” and “perhaps.”

How can we nonexperts test our own hunches?

‘Monitor your tone and thoughts. If you’re being swept away with enthusiasm for some particular course of action, take a deep breath and ask: Can I see anything wrong with this? And if you can’t, start worrying; you are about to go over a cliff.’

  • “Ironically, the more famous the expert, the less accurate his or her predictions tended to be.”
  • “ The less successful forecasters tended to have one big, beautiful idea that they loved to stretch, sometimes to the breaking point. They tended to be articulate and very persuasive as to why their idea explained everything.”
  • The unsuccessful build up momentum with “moreover” and “all the more so.” They are more entertaining. The media loves them.
  • “Like all of us, experts go wrong when they try to fit simple models to complex situations. (“It’s the Great Depression all over again!”) They go wrong when they leap to judgment or are too slow to change their minds in the face of contrary evidence.”


“Our confidence in a point of view should wax or wane with its predictive success and failures, the exact amounts hinging on the aggressiveness of forecasters’ ex ante theoretical wagers and on our willingness to give weight to forecasters’ ex post explanations for unexpected results.” – Philip Tetlock

Philip Tetlock, Expert Political Judgment

Further Reading : CNN

HistorySquared :

In short, think in terms of probabilities rather than certainties, and actively look for information to disprove your thesis.

Perhaps I should check my self on the uber bearish China thesis and look for the bullish side.  The problem is the closer I look, the worse the situation appears.

Related Posts Plugin for WordPress, Blogger...

Popularity: 2% [?]


Previous post:

Next post: