Nvidia CEO Jensen Huang says the performance of his company’s AI chips is advancing faster than historical rates set by Moore’s Law, the rubric that drove computing progress for decades.
“Our systems are progressing way faster than Moore’s Law,” said Huang in an interview with TechCrunch on Tuesday, the morning after he delivered a keynote to a 10,000-person crowd at CES in Las Vegas.
Coined by Intel co-founder Gordon Moore in 1965, Moore’s Law predicted that the number of transistors on computer chips would roughly double every year, essentially doubling the performance of those chips. This prediction mostly panned out, and created rapid advances in capability and plummeting costs for decades.
In recent years, Moore’s Law has slowed down. However, Huang claims that Nvidia’s AI chips are moving at an accelerated pace of their own; the company says its latest datacenter superchip is more than 30x faster for running AI inference workloads than its previous generation.
“We can build the architecture, the chip, the system, the libraries, and the algorithms all at the same time,” said Huang. “If you do that, then you can move faster than Moore’s Law, because you can innovate across the entire stack.”
The bold claim from Nvidia’s CEO comes at a time when many are questioning whether AI’s progress has stalled. Leading AI labs – such as Google, OpenAI, and Anthropic – use Nvidia’s AI chips to train and run their AI models, and advancements to these chips would likely translate to further progress in AI model capabilities.
Huang rejects the idea that AI progress is slowing. Instead he claims there are now three active AI scaling laws: pre-training, the initial training phase where AI models learn patterns from large amounts of data; post-training, which fine tunes an AI model’s answers using methods such as human feedback; and test-time compute, which occurs during the inference phase and gives an AI model more time to “think” after each question.
“Moore’s Law was so important in the history of computing because it drove down computing costs,” Huang told TechCrunch. “The same thing is going to happen with inference where we drive up the performance, and as a result, the cost of inference is going to be less.”
(Of course, Nvidia has grown to be the most valuable company on Earth by riding the AI boom, so it benefits Huang to say so.)
Nvidia’s H100s were the chip of choice for tech companies looking to train AI models, but now that tech companies are focusing more on inference, some have questioned whether Nvidia’s expensive chips will still stay on top.
AI models that use test-time compute are expensive to run today. There’s concern that OpenAI’s o3 model, which uses a scaled up version of test-time compute, would be too expensive for most people to use. For example, OpenAI spent nearly $20 per task using o3 to achieve human-level scores on a test of general intelligence. A ChatGPT Plus subscription costs $20 for an entire month of usage.
Huang held up Nvidia’s latest datacenter superchip, the GB200 NVL72, onstage like a shield during Monday’s keynote. This chip is 30 to 40x faster at running AI inference workloads than Nvidia’s previous best selling chips, the H100. Huang says this performance jump means that AI reasoning models like OpenAI’s o3, which uses a significant amount of compute during the inference phase, will become cheaper over time.
Huang says he’s overall focused on creating more performant chips, and that more performant chips create lower prices in the long run.
“The direct and immediate solution for test-time compute, both in performance and cost affordability, is to increase our computing capability,” Huang told TechCrunch. He noted that in the long term, AI reasoning models could be used to create better data for the pre-training and post-training of AI models.
We’ve certainly seen the price of AI models plummet in the last year, in part due to computing breakthroughs from hardware companies like Nvidia. Huang says that’s a trend he expects to continue with AI reasoning models, even though the first versions we’ve seen from OpenAI have been rather expensive.
More broadly, Huang claimed his AI chips today are 1,000x better than what it made 10 years ago. That’s a much faster pace than the standard set by Moore’s law, one Huang says he sees no sign of stopping soon.
