AI is hungry for computing power

The rise of artificial intelligence (AI) is driving up demand for computing power. However, AI data centres are being built faster than they can be connected to the grid, making power a crucial bottleneck.

Computing power is the driving force behind our digital economy and society. Especially in the United States, large data centres – basically supercomputers providing computing power – are springing up at a rapid pace. ‘The demand for computing power far exceeds supply’, says Joris Franck, Portfolio Manager and Technology Expert at KBC Asset Management.

The demand for computing power has been rising for decades: more powerful chips enable new applications, which in turn drive demand. ‘AI has now turbocharged this demand. It’s as if, instead of cars, we now all of a sudden have lorries driving on the same digital motorway.’

Building more data centres is not such a simple solution. ‘AI data centres are extremely capital-intensive and technically complex. Each new data centre pushes the boundaries of what is possible’, says Franck. ‘Moreover, they are fundamentally different from their predecessors. Traditional data centres run primarily on CPUs (Central Processing Units) and support applications such as websites, streaming and cloud software. AI data centres are built to train and utilise AI models.’

That difference is also reflected in the hardware. ‘AI data centres run primarily on GPUs (Graphics Processing Units) or AI accelerators. Engineers are making hundreds of thousands of GPUs work together in a single cluster, which could soon be millions. A million GPUs can easily cost 30 billion dollars. And that doesn’t even include the building, the power supply and the rest of the infrastructure.’

The physical scale of that infrastructure is mainly reflected in the energy demand. ‘The largest AI data centres are designed to continuously consume around one gigawatt’, says Jonas Theyssens, Portfolio Manager and industry expert at KBC Asset Management. ‘That’s about 24 gigawatt-hours a day, roughly equivalent to the daily consumption of around 750 000 households.’

‘In the US, for the first time in about 15 years, we’re seeing a clear increase in electricity demand again’, says Theyssens. ‘The combination of the electrification of cars and heat pumps, and of data centres as a new major consumer, is pushing demand structurally higher.’ Yet, according to Theyssens, the root of the problem does not lie in an absolute power shortage. ‘The fundamental problem is that electricity is a physical product that must be supplied locally and continuously. You can’t just move power around. And that is precisely where the problem lies: data centres are concentrated in specific regions and demand enormous amounts of power all at once.’

The result is a bottleneck in the electricity grid. ‘Historically, the grids aren’t built for such heavy and concentrated loads, let alone for the pace at which new projects are rolled out. This makes time-to-power crucial. A data centre can be built relatively quickly, in about one and a half to two years, but accessing power takes much longer. In some regions, waiting times for grid connection can be as long as seven years. The digital world thinks in quarters and the world of electricity thinks in years or even decades. Without a connection, even the most advanced data centre is just an empty box.’

Meanwhile, the next driver of demand is already on the horizon. ‘AI agents easily consume ten thousand times more computing power than a chatbot’, says Franck. ‘This involves software that performs tasks autonomously by controlling underlying AI models. These models process instructions and information as tokens: small pieces of text or data. Reports are already circulating of companies consuming their entire annual token budget in just a few months. In some cases, an AI agent can even end up being more expensive than a human employee.’