Forecasting Transformative AI, Part 1: What Kind of AI?

Abstract

The article argues that the development of AI systems capable of automating scientific and technological advancement, dubbed PASTA, could lead to an explosion in scientific progress and a radically unfamiliar future. The author explores how PASTA systems could be developed using modern machine learning techniques, drawing parallels between the training of AI systems like AlphaZero and the evolutionary process of humans. The author also discusses the potential dangers associated with PASTA systems, including the possibility of misaligned AI objectives and the potential for PASTA systems to surpass human capabilities, posing existential threats to humanity. The author concludes that the development of PASTA is a significant possibility in the coming decades and emphasizes the importance of understanding its potential impacts and risks. – AI-generated abstract