Disjunctive Scenarios of Catastrophic AI Risk

Abstract

This chapter analyzes various ways by which the development of sufficiently advanced artificial intelligence could pose an existential threat to humanity. It argues that focusing exclusively on the scenario of an AI achieving decisive strategic advantage, which would enable it to completely dominate the world, may be unwise, since an AI could also cause a catastrophe by achieving a major strategic advantage – a level of capability sufficient to pose a catastrophic risk to human society. The chapter considers various routes by which an AI may acquire a major or decisive strategic advantage, including individual takeoff scenarios via hardware overhang, speed explosion, and intelligence explosion; collective takeoff scenarios; scenarios where power slowly shifts over to AI systems; and scenarios in which an AI being good enough at some crucial capability gives it an MSA/DSA. It also examines the different ways in which an AI might gain the power to act autonomously, such as by escaping confinement or being intentionally released, and discusses the implications of having multiple AIs rather than just a single one. Finally, the chapter considers different combinations of these various routes to catastrophe and concludes that each of these scenarios will need to be evaluated for its plausibility, as well as for the most suitable safeguards for preventing it. – AI-generated abstract.