Of myths and moonshine

Stuart Russell

Edge.org, November 14, 2014

Abstract

The central problem with Artificial Intelligence (AI) is identified as the pervasive mythology surrounding it—the quasi-religious belief that algorithms are equivalent to sentient life, destined to surpass or destroy humanity. This narrative creates tangible negative consequences, including periodic disappointments in the technical field and a dangerous economic structure. Current big data AI relies heavily on uncompensated contributions from human users (e.g., translators providing corpora), creating the illusion of autonomous machine intelligence while functionally supporting a new technical elite, mirroring the dynamics of organized religion. Furthermore, the focus on speculative existential AI threats distracts from immediate, controllable dangers posed by technological actuators (such as weaponized drones), regardless of algorithmic sophistication. Contributing experts widely agree that fears of imminent, malevolent super-intelligence are exaggerated, noting that AI progress is not exponentially increasing in general cognition, and that the greater short-term risk lies in machine stupidity—systems making critical decisions they are not intelligent enough to manage. The consensus is that policy and ethical effort should be directed toward ensuring algorithms are provably aligned with human values, rather than fixating on hypothetical technological singularities. – AI-generated abstract.

PDF

First page of PDF

Quotes from this work

A system that is optimizing a function of n variables, where the objective depends on a subset of size k<n, will often set the remaining unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable. This is essentially the old story of the genie in the lamp, or the sorcerer’s apprentice, or King Midas: you get exactly what you ask for, not what you want.

p. 28