The problem of artificial suffering

Abstract

This is a comprehensive critical summary of Thomas Metzinger’s recent paper, Artificial Suffering: An Argument for a Global Moratorium on Synthetic Phenomenology. He thinks it’s our moral duty to minimize ENP risk: the risk of an ’explosion of negative phenomenology’ tied to conscious awareness in future machines. Metzinger is a professor of philosophy at the Johannes Gutenberg University Mainz and is notable for developing a theory of consciousness called the self-model theory of subjectivity. He recently served on the European Commission’s high-level expert group on artificial intelligence. Metzinger’s argument has deep implications for the ways EA and EA-adjacent groups think about s-risks, AI research, and whole brain emulation (e.g. Karnofsky’s digital people or Hanson’s ems). In this post, I will summarize Metzinger’s argument, define common terminology in the consciousness literature, and make some brief comments at the end. Metzinger subscribes to some version of suffering-focused ethics, but the argument still holds in most broadly consequentialist ethical theories.