Extremely rapid AI progress as key driver of risk?

Abstract

The emergence of highly intelligent AI systems raises concerns about risks to humanity, especially in cases of sudden and dramatic progress. This article assesses sources of strategic advantage for superintelligent AI, considering both gradual and discontinuous progress. It discusses several potential advantages, including in science, economic, military, and cooperation, and examines whether risks are still prevalent in scenarios with continuous progress. The article highlights the need for targeted work to reduce the likelihood of collusion among AIs. It argues that understanding and possibly avoiding intelligence explosions is crucial, and emphasizes the significance of continuity in deployed AI capabilities. The study underscores the role of leading AI developers in ensuring responsible progress. – AI-generated abstract.