Challenges in evaluating forecaster performance

Abstract

There are some good reasons to assess how good you or others are at forecasting (alongside some less-good ones). This is much harder than it sounds, even with a fairly long track record on something like Good Judgement Open to review: all the natural candidates are susceptible to distortion (or ‘gaming’). Making comparative judgements more prominent also have other costs. Perhaps all of this should have been obvious at rst glance. But it wasn’t to me, hence this post.