No Universally Compelling Arguments

Abstract

There is no universally compelling argument, i.e., no argument that any possible mind must accept. The intuition that such arguments exist arises from the flawed notion of a “ghost in the machine”—an irreducible core of reasonableness within a mind, separate from its physical implementation. This intuition is analogous to thinking of code as merely a suggestion to a computer, rather than the computer’s essence. From a physicalist perspective, any physical system that accepts an argument can be modified to reject it. The belief in universally compelling arguments leads to the search for a single moral principle that all AIs must inevitably conclude, a misguided approach to Friendly AI. Validity, worth, and rationality cannot be grounded in universal agreement, but must be defined within specific cognitive architectures. – AI-generated abstract.