At the Consumer Electronics Show (CES) in Las Vegas, Nvidia CEO Jensen Huang made significant claims regarding the performance advancements of his company’s AI chips, suggesting they are improving at a rate that exceeds the traditionally recognised benchmarks established by Moore’s Law. In an interview with TechCrunch on Tuesday, Huang stated, “Our systems are progressing way faster than Moore’s Law," underscoring his belief that Nvidia is setting a new standard in computing capability.
Moore’s Law, a term that originated from Intel co-founder Gordon Moore in 1965, posits that the number of transistors on a computer chip doubles approximately every year, leading to exponential improvements in performance and reductions in costs. However, in recent times, the momentum predicted by Moore has begun to slow. Contradicting this trend, Huang pointed out that Nvidia’s latest data centre superchip has achieved performance levels more than 30 times faster for executing AI inference workloads than its predecessors.
Huang explained the company’s strategic advantage: “We can build the architecture, the chip, the system, the libraries, and the algorithms all at the same time,” a process he believes enables Nvidia to innovate beyond the constraints of Moore’s Law. Over recent months, discussions in the AI community have raised questions regarding the pace of AI development, with some experts expressing concern that progress may be faltering. However, Huang maintains that rather than slowing down, AI development is propelled by three primary scaling laws: pre-training, post-training, and test-time compute.
In an earlier podcast from November, Huang suggested that the AI sector was on the verge of experiencing “hyper Moore’s Law,” a concept that manifests as AI models scale effectively while reducing costs. During his keynote speech, he reiterated, “Moore’s Law was so important in the history of computing because it drove down computing costs. The same thing is going to happen with inference where we drive up the performance, and as a result, the cost of inference is going to be less.”
Nvidia’s cutting-edge H100 chips have been favoured by technology companies to train AI models. However, the industry's increasing focus on inference has sparked discussions on whether Nvidia’s premium chips will maintain their lead. Huang sought to alleviate concerns by highlighting that although AI models requiring test-time compute are currently expensive, Nvidia’s GB200 NVL72 chip can provide a 30 to 40 times improvement in performance for AI inference tasks compared to the best-selling H100.
Huang explained that the performance boost from their new chip means that AI reasoning models like OpenAI’s o3, which demand substantial computational resources during inference, will become more cost-effective over time. “The direct and immediate solution for test-time compute, both in performance and cost affordability, is to increase our computing capability,” Huang noted. This emphasis on enhancing computing power is seen as vital to creating more affordable and efficient AI solutions in the long term.
The growing competence of AI chips has contributed to a significant decrease in the cost of AI services within the past year. Huang expects this trend of diminishing costs to persist, despite some of the initial AI models being recognized as fairly expensive, exemplified by OpenAI’s recent experiences. Huang further asserted that Nvidia’s current AI chips are now “1,000x better than what it made 10 years ago,” indicating a rapid acceleration far beyond that predicted by Moore’s Law.
The advances presented by Nvidia at CES and Huang's comments in particular reflect a notable evolution in AI technology. With the ability to enhance the architecture and algorithms simultaneously, Nvidia positions itself at the forefront of the burgeoning AI industry, suggesting a transformative period for business practices stemming from AI advancements.
Source: Noah Wire Services