Large Language Models (LLMs) have become all the rage since the release of ChatGPT in November 2022. The largest tech companies have jumped onto the bandwagon with their own offerings, from Google’s Bard and more recently Gemini, to Meta/Facebook’s Llama, thereby starting a race for the most advanced, capable model.

There has been no shortage of discussion about the implication of this emerging technology termed ‘Generative Artificial Intelligence (GenAI).From the ethical implications to privacy concerns, experts and non-experts alike have weighed in on topics such as whether it’s going to take all our jobs or whether it’s going to become sentient and bring an end to civilization. (Spoiler: it won’t).

Training LLMs

However, one of the lesser discussed implications of LLMs is the effect that they have on the environment. The computational resources needed to train and operate LLMs are extremely large. To illustrate this, GPT-3 (the model that powered ChatGPT at the time of release in November 2022) took 15 days to train and required 10,000 GPUs with almost 30,000 processing units [1]. This much computation consumes a significant amount of power which is one of the biggest contributors to the carbon footprint, and therefore environmental impact, of LLMs.

An academic paper written in 2020 estimates that training a large model, such as GPT-3, could produce as much carbon dioxide (CO2) as 5 cars over their lifetimes [2], which is a significant amount. This is a concern, given the rate of development and quantity of LLMs. GPT-3, for example, has already been superseded 3 times in less than 2 years, without even considering the multitude of other LLMs on the market.

And the troubling thing is that the training of the LLMs is not even the most problematic phase, in terms of the environmental impact.

 

Operating LLMs

Did you know that the inference phase, which is simply when the model is deployed after training and put into operation, could cost, computationally, 25x as much as the training of the model per year for an application like ChatGPT (3)?

For something like the Google search engine, which is shifting to use LLMs as the underlying mechanism, it could cost as much as 1,400x the training cost per year. Another issue with this is that the inference stage is significantly less studied than the training. With that in mind, a grain of salt should be taken with the above numbers but nonetheless it demonstrates a need for refocusing of research.

 

What is all the water for?

Another environmental aspect to consider is the water consumption of LLMs throughout their lifecycle.

There are two components to this. The first is the water used for the generation of power, which is in turn consumed in the data centers. Second is the water used for cooling in the data centers.

Using ChatGPT as an example (I keep picking on ChatGPT for no other reason than there is the most public information about it, being the first LLM to be widely available to the public), the training phase consumed an estimated 700,000 litres of water, equivalent to the amount of water consumed by a household over 5 years! As for the inference phase, it is estimated to use 500 ml of fresh water for every 20–50 requests [4].

With the context that ChatGPT receives over 100 million users per month, one can imagine the number of questions that are asked to it.

 

Addressing LLMs with sustainable solutions

As LLMs continue to grow in popularity and more companies begin to develop their own models, these environmental concerns will only be compounded. Yet, there is reason to be optimistic as efforts are being made to limit the environmental impact of LLMs, and AI in general.

 

Building awareness

Awareness is the first, and often hardest, obstacle to overcome. After all, you can’t solve a problem that you don’t know exists. There is a growing body of research on the topic of the environmental impact of AI, including the specific impact of LLMs. Organizations are also becoming more cognizant of the effect that AI has on the environment and while there is certainly more to be done, spreading awareness and accurate information is a good start.

 

Taking action

On a more practical front, the companies like Google, Microsoft and Meta that operate the data centers where these LLMs are run are trying to improve the sustainability of these facilities, which will in turn limit the environmental impact of anything run on those servers. These efforts include using sustainable, green sources such as solar or wind to power data centers and using less fresh water for cooling by implementing alternate cooling measures. This reduces wastewater, thus having a positive effect on the environmental footprint of the facilities.

In addition to this, manufacturers of hardware commonly used for model training and inference are working to improve the efficiency of their devices, and specifically the efficiency of AI-related tasks. A slightly extreme example of this is the Tensor Processing Unit (TPU), developed by Google for the specific task of running AI workloads. Alongside the hardware, the software —  algorithms and/or architecture —  are being improved to be more computationally efficient, leading to a reduced environmental impact of training and operating models.

 

Sustainable software development

At Eviden, we prioritize the impact of digital technologies on the environment. In April we released The Eviden Handbook on Sustainable Software Development, a guidebook on the environmental considerations that teams should be making when developing software solutions.

This includes best practices and questions we should be asking ourselves during planning and development, as well as key topics, such as sustainable data center operations. Without a doubt, this extends to the development of AI solutions, pointing back to the points made earlier about awareness and cognition, and making data centers more environmentally friendly.

As the European leader and a global player in supercomputers, we also put sustainability at the heart of our hardware strategy. Our BullSequana supercomputers are designed to be extremely efficient and are regularly featured on the Green500 list of the most sustainable HPC systems in the world. In June 2024, the Eviden-technology based JEDI – JUPITER Exascale supercomputer from EuroHPC/FZJ in Germany has notably been ranked number one in the 2024 Green500. The HPC system will probably be the most powerful AI system worldwide and will also be used to train LLMs.

Despite the environmental concerns, it is important not to lose sight of the many benefits that AI can have for the environment, namely in optimization of processes that result in lower carbon emissions, such as in logistics and transport, in monitoring and remote sensing, in climate modelling and prediction, and in recycling. LLMs also play their part by making it easier to access information on environmental topics, such as climate change, thus improving people’s education on these important matters.

 

Weighing Innovation and Environmental considerations

During the recent surge of LLMs, the environmental repercussions have often been overlooked. The immense computational resources required for training and operating LLMs result in a significant carbon footprint, with estimates likening training emissions to those of multiple cars over their lifetimes. The inference phase, often disregarded, only exacerbates this impact.

Water consumption throughout the lifecycle of LLMs is also alarming, from power generation to cooling in data centers. Without intervention, these environmental concerns will only escalate as LLMs become more widespread. But there is reason not to be too cynical. Efforts to mitigate LLMs’ environmental impact are underway, including raising awareness, enhancing data center sustainability, and improving hardware and software efficiency. By prioritizing sustainability alongside technological advancement, we can move towards a more environmentally responsible future for AI innovation.

References

Related Articles

View all blogs