The swift emergence of artificial intelligence models such as ChatGPT and Gemini raises fundamental questions about their true training cost. Recent technological advancements come with staggering expenses and a concerning ecological footprint. Leading this reflection involves a meticulous analysis of the financial stakes, where estimates run into millions, if not billions of dollars. The need to optimize these resources becomes pressing in the face of growing environmental challenges, involving both innovative technologies and sustainable practices.
Training AI Models: An Exponential Cost
The costs associated with training artificial intelligence models, such as ChatGPT and Gemini, are experiencing a dramatic increase. The first generations of these models required investments of a few million dollars. Today, these sums have exploded, reaching hundreds of millions, or even billions for upcoming models.
According to an estimate by Dario Amodei, CEO of Cohere, the current training cost stands at 100 million dollars. Some models in development could cost up to one billion dollars. Amodei predicts that expenditures could reach dizzying heights, amounting to between ten and one hundred billion dollars in the near future.
A study by Epoch AI highlights that the training costs of artificial intelligence models have risen at an impressive rate. Since 2016, these costs have been increasing at a factor of 2.4 times per year. This price inflation could push smaller companies out of the market.
Specific Training Costs of AI Models
The training cost of the GPT-4 model, introduced by OpenAI in 2023, is estimated to be over 100 million dollars. Meanwhile, the training of the Gemini 1 model is reported to have required an investment of nearly 191 million dollars from Google. These figures include the acquisition of GPUs, which have unit prices ranging from 30,000 to 40,000 dollars, as well as cloud infrastructure expenses.
Financial Implications and the AI Bubble
David Cahn, an analyst at Sequoia Capital, warns about the risk of a financial bubble. To balance investments in the sector, large companies need to generate around 600 billion dollars in annual revenue. Assuming they each make 10 billion dollars through artificial intelligence, there would remain a gap of 500 billion dollars to fill.
The colossal costs create increased pressure on companies, which must explore optimization techniques to reduce training durations for models. The JEST method, developed by Google DeepMind, could reduce the necessary calculations by up to ten times.
Environmental Impact and Energy Consumption
Concerns about the environmental impact of training AI models are intensifying. Energy consumption relies on immense needs, requiring innovative cooling solutions to prevent server overheating. The Jean Zay supercomputer, located in France, uses a water cooling system, reusing heated water for surrounding housing.
Furthermore, the water footprint presents a challenge that is often underestimated. An American study indicates that asking 20 to 50 questions to GPT-3 consumes a volume of water equivalent to a 50 cl bottle used for cooling servers. This consumption could reach 4 to 6.5 billion cubic meters of freshwater by 2027, representing a substantial withdrawal comparable to that of Denmark.
Initiatives for Better Sustainability
In light of this issue, some companies, like Microsoft, are adopting new energy strategies. In 2024, the tech giant signed an agreement to reactivate the Three Mile Island nuclear power plant to power its data centers by 2028. This decision sparks debate, as it comes while Microsoft is working to reduce its CO2 emissions.
Regulating AI for a Sustainable Future
Industry players are striving to establish sustainable practices for training AI models. Solutions such as water cooling, innovated by Atos, help limit water consumption while maintaining energy efficiency. This technique could significantly reduce energy costs by about 20 to 30%.
A decree recently signed by American President Joe Biden aims to facilitate access to resources for researchers, in order to avoid a concentration of AI developments in the hands of a few companies. Increased transparency regarding the carbon and water footprint of AI models could emerge, promoting a more responsible approach.
The issues related to energy consumption, water footprint, and extra costs remain pressing in the quest for a balance between technological innovation and sustainable development, placing the sector at the heart of a necessary debate.
Frequently Asked Questions
What is the training cost of an AI model like ChatGPT?
The training cost of a model like ChatGPT is estimated to be over 100 million dollars, due to the needs for hardware and cloud infrastructure.
Why has the cost of training AI models increased so much in recent years?
The increase in costs is explained by the rising demands for computing power, particularly the use of expensive GPUs and the inflation of infrastructure costs.
What is the environmental impact associated with training models like Gemini?
The training of models like Gemini has a significant environmental impact, with high energy consumption and a concerning water footprint, which could reach several billion cubic meters of freshwater in a few years.
How are companies trying to reduce the training costs of AI?
Companies are exploring optimization techniques, such as Google’s JEST method, which could reduce the necessary calculations by up to ten times, as well as more efficient cooling methods.
What costs are included in the training of an AI model?
Training costs include the purchase of hardware like GPUs, expenses related to cloud infrastructures, and other operational costs such as energy and server cooling.
Is there a financial bubble around AI investments?
Some analysts, like David Cahn, warn of a potential financial gap, emphasizing that large companies must generate substantial revenue to balance their investments.
Are training costs for AI models transparent?
Often, training costs are not publicly shared by companies, making it difficult to obtain a complete picture of actual spending.
Which AI models are currently the most expensive to train?
Recent models like GPT-4 and Gemini 1 are among the most expensive, with estimates exceeding 100 million dollars for the former and nearly 191 million for the latter.
Can the carbon impact of AI training be measured?
Yes, studies have shown that training a model like GPT-3 released the equivalent of 502 tons of CO2, highlighting the importance of considering the carbon footprint of AI models.
What initiatives are being taken to make AI training more sustainable?
Initiatives include research on more efficient cooling methods and efforts to diversify the energy sources used during model training.
