The significant rise in the total cost of ownership (TCO) of datacentres has emerged as a critical concern for businesses aiming to harness the power of artificial intelligence (AI). According to Phil Burr, Director at Lumai, several factors are fuelling this increase, largely tied to the escalating energy demands brought on by the AI sector. In a recent discussion with Intelligent Data Centres, Burr proposed that innovative technologies like 3D optics could play a pivotal role in addressing these challenges.
The UK National Grid has issued warnings about the impending surge in power usage attributed to the expansion of AI, predicting that datacentre power consumption could experience a six-fold increase over the next decade. Concurrently, Goldman Sachs anticipates that the tech industry will soon exceed $1 trillion in spending on datacentres and hardware, underscoring a stark reality: this spending spree, while beneficial for innovation, creates a considerable strain on power resources both at the datacentre and rack levels.
One of the primary reasons for the increase in TCO is the burgeoning demand for datacentre capacity driven by AI advancements. In a recent study published by McKinsey, it was revealed that global demand for datacentre capacity could rise annually by between 19 and 22 per cent from 2023 to 2030. Notably, AI-ready datacentre capacity demand is projected at 33%, with generations of AI (GenAI) representing an even higher figure of 39%. As major cloud service providers strive to meet this demand, they are not only constructing new datacentres but are also collaborating with colocation providers. This shift has led to a noticeable rise in costs, as colocation prices rose by an average of 35% between 2020 and 2023, reversing the previous trend of declining prices.
Burr points out that merely reusing or upgrading existing infrastructure will not suffice to accommodate the skyrocketing demand. Instead, the sector must innovate in AI computation to decrease energy usage, enabling processing to stay within established power limits. "Given the power capacity constraints and the insatiable demand, it is clear that now is the time to look at these different approaches," Burr explained.
The quest for efficiency has been hindered by the limitations of silicon chip-based accelerators, which struggle to meet the sophisticated and growing capabilities necessary for AI processing within practical power boundaries. McKinsey's report highlights an alarming increase in average power densities in datacentre racks, which have more than doubled in just two years. Projections estimate that by 2027, power densities could reach as high as 30kW per rack. AI models such as ChatGPT can reportedly consume upwards of 80kW per rack, with leading chips from manufacturers like Nvidia demanding densities of up to 120kW.
Burr advocates for a shift towards optical AI acceleration, suggesting that the use of photons for computation could radically improve energy efficiency, enabling a level of performance that current silicon technology cannot achieve. He noted that optical AI accelerators could operate using only 10% of the power consumed by conventional GPUs, while still significantly enhancing performance. This transition to optical systems not only aims to lower power consumption but could also extend the operational lifespan of existing datacentres and diminish the necessity for new facilities, resulting in a marked decrease in TCO.
As businesses grapple with escalating hardware costs, enhanced performance expectations are driving manufacturers to continuously pour resources into silicon-based solutions. Nvidia, for example, recently cited the costs associated with its Blackwell GPU, estimated at between $30,000 and $40,000, alongside development costs soaring to $10 billion. Burr contended that a pivot towards cost-effective optical technology could alleviate this fiscal pressure, reducing reliance on expensive new silicon technology.
Reflecting on the historical context, Burr noted that between 2015 and 2019, even as workloads tripled, datacentre power demand was kept relatively steady due to a concerted focus on efficiency. While the challenge posed by AI may be greater, this precedent illustrates the possibilities that lie in innovative practices aimed at reducing costs and enhancing sustainability.
In conclusion, the industry is at a crossroads where the sustainability of AI advancement and the economic feasibility of datacentre operations are becoming increasingly intertwined. The current trajectory regarding TCO for AI datacentres is unsustainable. However, with the advent of technologies such as optical AI acceleration, there exists the potential for businesses to not only lower their TCO but also to contribute positively to environmental sustainability while catering to the surging demand for AI capabilities.
Source: Noah Wire Services