Effective altruism, risk, and human extinction

Abstract

A future in which humanity does not go extinct from something like a meteor strike or nuclear war might contain vast quantities of great happiness and human flourishing. But it might also contain vast quantities of great misery and wasted potential. Any effort to ensure a long happy future will have to strive both to ensure a long future, by reducing extinction risks, and to ensure it is a happy one, by improving the quality of life. Such an effort might succeed at the former goal without succeeding at the latter. So any effort to ensure a long happy future will increase the chance of such a future, but it will also increase the chance of a long miserable future, even if it increases the latter by a smaller amount. Granted this, if you are risk-averse, or if morality requires you to choose using risk-averse preferences, then you do better to work to bring about humanity’s extinction than to secure its long-term survival. This conclusion will seem troubling to many, though perhaps welcome to some. In this paper, I try to formulate the argument as plausibly and as robustly as I can. I then investigate how those who wish to resist it might do so.