Thoughts on the singularity institute (SI)

Abstract

Holden Karnofsky, Co-Executive Director of GiveWell, expresses concerns about the Singularity Institute (SI), a charity GiveWell has been asked to evaluate. He argues that SI’s claims about the importance of its work are both wrong and poorly argued, potentially increasing the risk of an AI-related catastrophe. Karnofsky also criticizes SI’s organizational structure and personnel, questioning their suitability for the task at hand. He rejects the common argument that even a small chance of SI being right justifies supporting it, stating that SI representatives themselves advocate for supporting the organization only if it presents strong arguments and operates effectively. While GiveWell currently does not recommend SI, Karnofsky acknowledges that these views are open to change and that SI is working to address his concerns.