Pascal's mugging: tiny probabilities of vast utilities

Abstract

This work presents the Pascal’s Mugging problem, which consists of a vastly improbable but enormous-consequence threat made by a seemingly unreliable agent, whose low probability is outweighed by the high value of the threatened outcome. The article proposes that complexity-based priors implemented by Solomonoff induction solve this problem by making estimates that threats about enormous outcomes are massively unlikely. Even if the threat is utterly credible, a sensible agent should not get Pascal-mugged because it is still more probable the threatening scenario would not arise at all. Variants of this problem are proposed, such as the threat of inflicting large yet finite disutility, or changing the experiment to see what differential behavior is elicited. The article concludes by asking how agents should avoid being dominated by tiny probabilities of vast utilities, as this line of reasoning could be exploited by Pascal’s Muggers. – AI-generated abstract.