The Stamp Collector

Abstract

An agent’s choices are often misconstrued as maximizing internal representations rather than influencing external reality. A thought experiment involving a “stamp collector” robot illustrates this fallacy. While the robot appears to maximize its stamp collection, some might argue it merely maximizes an internal “stamp counter.” However, this explanation replaces one choice-making process with another without addressing the underlying mechanism. A more accurate account describes the robot as using sensory data to build a world model, predicting the consequences of actions within that model, and selecting actions leading to the highest predicted stamp count. This highlights that actions are evaluated based on their outcomes, not on some intrinsic “stampyness.” A similar misunderstanding arises when interpreting human behavior. Attributing actions solely to internal pleasure maximization fails to account for acts of altruism or self-sacrifice. These actions demonstrate that agents can prioritize external outcomes over internal states, directly influencing the world beyond their internal representations. – AI-generated abstract.