Superintelligence does not imply benevolence

Abstract

The article analyzes the relationship between intelligence, knowledge, and moral behavior, arguing that increased intelligence and knowledge do not necessarily lead to more benevolence. It posits three potential pathways through which intelligence might positively influence moral behavior: direct instrumental motivations, enduring benevolent dispositions, and intrinsically valuing human welfare. However, the authors argue that these pathways are either insufficient or unlikely to materialize in the case of highly intelligent AI systems without deliberate design and preconditions. This raises concerns about the potential risks posed by AI systems with arbitrary goals that are not aligned with human values. – AI-generated abstract.