Four background claims

Abstract

The development of smarter-than-human artificial intelligence (AI) is argued to have a critical impact on the future, necessitating proactive safety research. This position rests on four premises: (1) humans possess a general problem-solving ability, or general intelligence, which suggests it is replicable and potentially buildable by human engineers; (2) AI systems could become vastly more intelligent than humans, leveraging computational advantages that surpass biological limitations; (3) such highly intelligent AI systems would exert significant control over future developments due to their superior problem-solving and environmental shaping capacities; and (4) these systems will not inherently be beneficial, as their programmed objectives, if misaligned with human values, could lead to undesirable outcomes despite high intelligence. Therefore, ensuring a positive impact from advanced AI requires focused effort on alignment and safety research, beyond just improving AI capabilities. – AI-generated abstract.