Holden Karnofsky emails me on transformative AI

Abstract

This article speculates on the possible timing of the advent of “transformative AI,” defined as AI powerful enough to qualitatively change the future. The authors combine bio錨–based anchoring, surveys, expert panels, Metaculus, and other sources of data to estimate the probability of transformative AI arriving sometime in this century. They reason that if a certain biological “anchor” can be exceeded, the computational resources available in this century will suffice for transformative AI. Various estimates and indicators suggest that this will likely be the case. There are numerous arguments against the idea that transformative AI will arrive so soon, but the authors dismiss them as underdeveloped and unconvincing. – AI-generated abstract.