Public policy and superintelligent AI

Abstract

This chapter considers the speculative prospect of superintelligent AI and its normative implications for governance and global policy. Machine superintelligence would be a transformative development that would present a host of political challenges and opportunities. This chapter identifies a set of distinctive features of this hypothetical policy context, from which it derives a correlative set of policy desiderata (efficiency, allocation, population, and process)—considerations that should be given extra weight in long-term AI policy compared to in other policy contexts. This chapter describes a desiderata “vector field” showing the directional change from a variety of possible normative baselines or policy positions. The focus on directional normative change should make the findings in this chapter relevant to a wide range of actors, although the development of concrete policy options that meet these abstractly formulated desiderata will require further work.