Comparing top forecasters and domain experts

Abstract

The paper investigates the accuracy of superforecasters, a group of highly skilled individuals who have consistently demonstrated superior forecasting abilities, in comparison to domain experts and other forecasting mechanisms. The authors systematically review existing studies, focusing on claims that forecasters outperform (1) the general public, (2) simple models, and (3) domain experts. While finding strong evidence for claims (1) and (2), the paper reveals a surprising lack of robust evidence for claim (3). The authors conclude that, despite a common misconception, superforecasters have not demonstrably outperformed intelligence analysts with classified information. Further, they argue that while superforecasters may perform comparably to experts in certain domains, it might be more advantageous to focus on highly skilled ML professionals who possess both domain expertise and forecasting abilities, especially in complex domains like machine learning. – AI-generated abstract