Aligning superintelligence with human interests: a technical research agenda

Abstract

This technical agenda argues that there is foundational research approachable today that will make it easier to design aligned smarter-than-human systems in the future. It describes ongoing work on problems relevant to this goal, including formalizing the problem of computer intelligence, developing reliable, error-tolerant agent architectures, and addressing the challenge of value learning. The authors contend that a better understanding of these foundational problems is necessary to develop highly reliable, error-tolerant, and aligned smarter-than-human systems. They also argue that work on these problems should begin now, even though practical smarter-than-human systems are still some time away. – AI-generated abstract.