AGI safety from first principles

Abstract

Modern machine learning threatens to create artificial general intelligence (AGI) and this could be an existential risk for humanity. The argument in this report, which argues for the imperative of safety during development of AGI, draws on first principles and is more comprehensive than previous arguments. It claims that AGI presents risks for which it is difficult to devise control mechanisms, and that the risk stems from recursive self-improvement. There is no sure approach to guard against this, though various possible means for assuring the safety of AGI are discussed. – AI-generated abstract.