About MIRI
Machine Intelligence Research Institute, 2021
Abstract
The article discusses the challenges in developing safe and reliable artificial intelligence (AI) systems, particularly in the context of general-purpose AI with human-equivalent intelligence or greater. It emphasizes the need for research on the mathematical underpinnings of intelligent behavior to enable the clean design and analysis of such systems. The article suggests that AI systems may be vulnerable to causing significant harm due to poor environmental models, incorrectly specified goals, and brittle decision-making. It argues for integrating robustness and safety considerations into mainstream AI capabilities research, drawing inspiration from fields like civil engineering and nuclear fusion. The article stresses the importance of breaking down the alignment problem into simpler subproblems and developing basic mathematical theory to inform engineering applications. – AI-generated abstract.
