Computation hazards

Abstract

Several ideas have been circulating LessWrong that can be organized under the concept of computation hazards: large negative consequences that may arise from vast amounts of computing. For example, suffering in simulated people, control over a user by a predictor oracle, and unboxing of powerful agents are all detailed. The concept of computational hazards is distinguished from other types of AGI risks. Finally, possible safety practices to avoid these hazards, such as limiting complexity or developing safety indicators, are mentioned, though the author concludes that writing a nonperson predicate—a function that can evaluate an algorithm and determine whether it is a person—may be FAI-complete. – AI-generated abstract.