works
Daniel Kokotajlo The date of AI Takeover is not the day the AI takes over online The key concept regarding AI timelines isn’t when AI takes over, but when it becomes too late to mitigate AI risk – the point of no return. This could occur long before AI reaches world-dominating power. Several scenarios illustrate this: unaligned superhuman AI nearing deployment with no alignment solution and imminent replication by others, precluding intervention; narrow AIs integrated into the economy driving a slow takeoff, outpacing safety/alignment research while economic and political pressures prevent course correction; and advanced persuasion tools fracturing collective epistemology, hindering serious consideration of AI risk due to entrenched ideological factions. Therefore, predicting AI takeover should focus on when our influence over the future significantly diminishes, marking the point of no return.

The date of AI Takeover is not the day the AI takes over

Daniel Kokotajlo

LessWrong, October 21, 2020

Abstract

The key concept regarding AI timelines isn’t when AI takes over, but when it becomes too late to mitigate AI risk – the point of no return. This could occur long before AI reaches world-dominating power. Several scenarios illustrate this: unaligned superhuman AI nearing deployment with no alignment solution and imminent replication by others, precluding intervention; narrow AIs integrated into the economy driving a slow takeoff, outpacing safety/alignment research while economic and political pressures prevent course correction; and advanced persuasion tools fracturing collective epistemology, hindering serious consideration of AI risk due to entrenched ideological factions. Therefore, predicting AI takeover should focus on when our influence over the future significantly diminishes, marking the point of no return.

PDF

First page of PDF