AI Energy Consumption

AI energy consumption measures how much electricity AI uses. Training a large model can draw power from thousands of accelerators for weeks, and serving AI to millions of users requires large fleets of GPUs running continuously. Each AI request uses a small amount of energy, but the total adds up quickly at scale, and reasoning models and agents use more energy per task because they process more tokens. Cooling, networking and power conversion in the data center add to the energy drawn by the chips themselves.

The International Energy Agency estimates that data centers use around 1.5% of the world's electricity and expects that share to grow substantially by the end of the decade, with AI as a major driver. For organizations, AI energy consumption matters in three ways: as a cost, either directly for owned hardware or embedded in the prices of providers; as a constraint on how quickly AI capacity can be added; and as a source of emissions that affects sustainability commitments. For IT financial management, energy is becoming a material part of the economics of AI.

Definition of AI Energy Consumption

AI energy consumption is the electrical energy used by computing infrastructure to train, fine-tune and run AI models, plus the share of supporting facility energy, such as cooling, attributable to that infrastructure.

It can be measured for a facility, a cluster, a workload or a unit of work, such as energy per training run, per thousand requests or per million tokens generated. Measures per unit of work are the most useful for comparing options, since they account for both efficiency and volume.

Where the Energy Goes

Accelerators such as GPUs account for most of the direct energy use. High-end AI accelerators draw far more power than conventional server processors, and racks of them draw many times the power of traditional racks.

Cooling removes the heat generated by the hardware and can add substantially to total energy use, depending on the efficiency of the facility and the climate.

Networking, storage and power distribution account for the rest. Data center efficiency is often summarized as power usage effectiveness, the ratio of total facility energy to IT equipment energy.

Training and Inference

Training large models uses large amounts of energy in concentrated periods. Fine-tuning uses far less.

Inference uses smaller amounts per request but runs continuously and grows with usage, so for widely used models it often accounts for most of their lifetime energy use.

The energy per request varies greatly with model size, the number of tokens processed and the efficiency of the hardware and serving software. A small model answering a short question can use a tiny fraction of the energy of a large reasoning model working through a complex task.

Importance of AI Energy Consumption in IT Financial Management

Energy is a significant operating cost for AI infrastructure. For owned or colocated hardware, electricity appears directly in operating budgets. For cloud and API services, it is built into prices.

Power availability limits the pace at which new AI data centers can be built, which affects the supply and price of AI capacity for everyone.

AI energy use also affects reported emissions. Organizations with climate commitments need to account for the energy their AI workloads consume, including through suppliers. This makes AI energy part of the same picture as cloud sustainability.

Energy as a Cost

For self-hosted AI, energy cost per unit of work depends on hardware efficiency, utilization and local electricity prices. Idle GPUs still draw power, so low utilization raises energy cost per unit of work as well as hardware cost.

Newer accelerators usually deliver more work per unit of energy, which can make upgrading attractive even before older hardware reaches the end of its useful life.

Energy costs belong in the total cost of ownership of any owned AI infrastructure.

Energy and Emissions

The emissions from AI depend on the carbon intensity of the electricity used, which varies widely by region and time of day.

Providers increasingly report energy and emissions data, but methods differ, and estimates for API usage are often approximate. Organizations should be clear about the method used when reporting AI emissions.

Choosing efficient models, reducing unnecessary tokens and using lower-carbon regions where possible reduce both cost and emissions.