What does it take to defend the world against out-of-control AGIs?

Abstract

Powerful good artificial general intelligences (AGIs) would fail to defend against out-of-control bad AGIs in many scenarios due to imbalances between offense and defense, lack of human trust in good AGIs, resource constraints, and the need to prevent bad AGIs in the first place. These problems seem unlikely to be solved through traditional approaches such as better cybersecurity, global coordination, or societal resilience. Instead, the only feasible path to preventing out-of-control AGIs may involve implementing extreme measures such as forcefully preventing the creation of all AGIs, using friendly but non-corrigible AGIs to defend against bad AGIs, or implementing a massive-scale defense system against all AGIs in the aftermath of an AGI-related catastrophe. While some of these measures may be technically possible, their likelihood of implementation seems low due to factors such as the difficulty of building safe and sufficiently powerful AGIs, the reluctance of responsible actors to use AGIs aggressively, and the potential for disastrous consequences if these measures are not implemented perfectly. – AI-generated abstract.