In a significant move aimed at reshaping the landscape of artificial intelligence (AI) hardware, Amazon is intensifying its efforts to rival Nvidia's dominant market position. With Nvidia currently commanding approximately 80 per cent of the AI hardware market, Amazon’s initiative represents a crucial step towards establishing its credentials in the AI ecosystem while simultaneously mitigating reliance on Nvidia's graphical processing units (GPUs).
At the forefront of Amazon's strategic shift is Annapurna Labs, a chip design startup that the company acquired in 2015 for $350 million. Over the years, Annapurna Labs has made notable advancements in custom chip production, specifically with the Graviton series targeted at traditional data centres and the Trainium series designed for AI applications.
The latest product from Amazon, Trainium 2, was unveiled during the AWS re:Invent event in November 2023. This new chip is designed in response to the increasing demand for specialised processors capable of training large language models and managing complex AI tasks efficiently. Amazon has asserted that Trainium 2 increases training performance by up to four times compared to the first-generation Trainium chips, offering substantial improvements in terms of speed, cost, and energy efficiency. According to Amazon, these chips can be integrated into EC2 UltraClusters, with the capacity to scale up to 100,000 chips. This capability is expected to drastically enhance the training of foundation models and large language models, thereby improving overall operational efficiency.
The initial deployment of Trainium 2 has begun with several organisations, including Anthropic—an Amazon-backed competitor to OpenAI—alongside Databricks, Deutsche Telekom, Ricoh, and Stockmark. The company plans to provide additional details about the Trainium 2 chips and its future developments in the AI hardware realm at an upcoming event.
Economic considerations play a crucial role in Amazon's AI strategy. The global AI sector has become increasingly dependent on Nvidia’s GPUs, which are widely regarded as the benchmark for executing complex AI workloads. However, this dependency is fraught with challenges due to the high demand and limited supply of Nvidia's GPUs, leading to increased costs for cloud service providers and their customers. In this context, Dave Brown, Amazon's vice president of compute and networking services, stated, "We want to be absolutely the best place to run Nvidia. But at the same time, we think it’s healthy to have an alternative."
Through the introduction of Trainium 2 and other custom AI chips, Amazon aims to alleviate some of the financial strains experienced by AWS clients. The company claims that its Inferentia line of processors has already provided 40 per cent lower operating costs for generating responses from AI models.
Amazon’s push is aligned with a broader trend across the technology sector, as several other major companies, including Google, Meta, and Microsoft, are equally investing in proprietary AI chip development to reduce their reliance on Nvidia. Google, for instance, has recently launched its latest tensor processing unit (TPU), Trillium, which reportedly offers four times faster AI training and three times faster inference than previous models. Meta has also introduced its next-generation Meta Training and Inference Accelerator (MTIA) this year.
While Amazon's drive into custom AI hardware is ambitious, the Trainium 2 chip has yet to achieve widespread adoption, with its availability limited post-launch in 2023. Experts are eagerly anticipating independent performance metrics that will allow for comparisons between Trainium 2 and Nvidia’s GPUs in practical applications.
Amazon's venture into developing custom AI chips is not only a pivotal element of the company's larger AI strategy but also a calculated effort to capture a significant share of the burgeoning AI hardware market by offering a cost-effective alternative to existing solutions.
Source: Noah Wire Services