Comments on Carlsmith's 'Is power-seeking AI an existential risk?'

Abstract

The following are some comments I gave on Open Philanthropy Senior Research Analyst Joe Carlsmith’s Apr. 2021 “Is power-seeking AI an existential risk?”, published with permission and lightly edited. Joe replied; his comments are included inline. I gave a few quick replies in response, that I didn’t want to worry about cleaning up; Rob Bensinger has summarized a few of them and those have also been added inline.