Nathan Calvin on California’s AI bill SB 1047 and its potential to shape US AI policy

Abstract

California’s Senate Bill 1047 seeks to codify voluntary industry commitments into binding law by regulating frontier artificial intelligence models that require over $100 million in compute to train. To mitigate catastrophic risks—including the weaponization of chemical, biological, or nuclear technologies, severe cyberattacks on critical infrastructure, and autonomous systems evading human control—the legislation mandates that developers establish Safety and Security Plans, conduct annual third-party audits, and maintain reasonable care to prevent foreseeable critical harms. Although critics contend the bill will stifle open-source innovation and disadvantage startups, the regulatory threshold exempts smaller models and protects downstream open-source users. Standard tort liability principles apply only to negligent developers whose models cause substantial mass casualties or financial damage, rather than imposing strict liability for all downstream misuse. Given the historical delays in federal legislative action, state-level policy in economically significant jurisdictions serves as a highly effective mechanism for shaping national and global industry standards, leveraging market size to mandate public safety compliance without halting technological progress. – AI-generated abstract.