Training Compute of Frontier AI Models Grows by 4-5x per Year

Abstract

Our expanded AI model database shows that the compute used to train recent models grew 4-5x yearly from 2010 to May 2024. We find similar growth in frontier models, recent large language models, and models from leading companies.