Gradual disempowerment

Abstract

The proliferation of advanced AI systems may lead to the gradual disempowerment of humanity, even if efforts to prevent them from becoming power-seeking or scheming are successful. Humanity may be incentivised to hand over increasing amounts of control to AIs, giving them power over the economy, politics, culture, and more. Over time, humanity’s interests may be sidelined and our control over the future undermined, potentially constituting an existential catastrophe.