Christoph Winter on the Legal Priorities Project

Abstract

Evidence-based policy, a popular approach that emphasizes the importance of data-driven decision-making, has been influenced by Randomized Control Trials (RCTs), particularly in development economics. However, there are challenges to interpreting RCT results, such as distinguishing between internal validity (the ability to get an unbiased estimate) and external validity (the ability to predict results outside of the sample). While internal validity is necessary, it is not sufficient. External validity can vary significantly across interventions, and it is difficult to generalize results from one study to another, leading to unreliable predictions. Researchers can improve external validity by considering factors such as baseline rates, context, and scale-up effects. The findings suggest that a more nuanced approach to evidence-based policy is needed, one that recognizes the limitations of RCTs and seeks to incorporate other sources of information, such as forecasting, to inform decision-making. – AI-generated abstract.