Nvidia CEO Jensen Huang
Getty
Nvidia on Monday announced a new generation of artificial intelligence chips and software for running AI models. The announcement, made during Nvidia’s developer conference in San Jose, comes as the chipmaker seeks to solidify its position as the go-to supplier for artificial intelligence companies.
Nvidia’s stock price has quintupled and total sales have more than tripled since OpenAI’s ChatGPT kicked off the AI boom in late 2022. Nvidia’s high-end server GPUs are essential to the training and deployment of large AI models. Companies like it Microsoft AND Half they spent billions of dollars buying the chips.
The new generation of AI graphics processors is called Blackwell. The first Blackwell chip is called GB200 and will be available later this year. Nvidia is luring its customers with more powerful chips to spur new orders. Companies and software makers, for example, are still struggling to get their hands on the current generation of H100 “Hopper” and similar chips.
“Hopper is great, but we need bigger GPUs,” Nvidia CEO Jensen Huang said Monday at the company’s developer conference in San Jose, California.
The company also introduced revenue-generating software called NIM that will make it easier to implement artificial intelligence, giving customers another reason to stick with Nvidia chips over a growing field of competitors.
Nvidia executives say the company is becoming less of a mercenary chip supplier and more of a provider of platforms, like Microsoft or Apple, on which other companies can build software.
“The commercial salable product was the GPU, and the software was intended to help people use the GPU in different ways,” said Manuvir Das, Nvidia’s corporate vice president, in an interview. “Of course, we still do it. But what’s really changed is that now we really have a commercial software business.”
Das said Nvidia’s new software will make it easier to run programs on any Nvidia GPU, even older ones that may be better suited for implementing but not creating AI.
“If you’re a developer, you have a cool model that you want people to adopt, if you put it in a NIM, we’ll make sure it runs on all of our GPUs, so we reach a lot of people,” Das said.
Meet Blackwell, Hopper’s successor
Nvidia’s Grace Blackwell GB200 superchip, with two B200 graphics processors and an Arm-based central processor.
Every two years Nvidia updates its GPU architecture, unlocking a big leap in performance. Many of the AI models released in the past year were trained on the company’s Hopper architecture, used by chips such as the H100, announced in 2022.
Nvidia says Blackwell-based processors, like the GB200, offer a huge performance upgrade for AI companies, with 20 petaflops of AI performance versus the H100’s 4 petaflops. The additional processing power will allow AI companies to train larger and more complex models, Nvidia said.
The chip includes what Nvidia calls a “purpose-built transform engine to run transformer-based AI, one of the core technologies behind ChatGPT.
The Blackwell GPU is large and combines two separately manufactured dies into a single chip manufactured by TSMC. It will also be available as an entire server called the GB200 NVLink 2, which combines 72 Blackwell GPUs and other Nvidia parts designed to train AI models.
Amazon, Google, MicrosoftAND Oracle will sell access to the GB200 via cloud services. The GB200 pairs two Blackwell B200 GPUs with an Arm-based Grace CPU. Nvidia said Amazon Web Services will build a server cluster with 20,000 GB200 chips.
Nvidia said the system can implement a 27 trillion parameter model. It is much larger than even the largest models, such as GPT-4, which reportedly has 1.7 trillion parameters. Many AI researchers believe that larger models with more parameters and data could unlock new capabilities.
Nvidia did not provide a cost for the new GB200 or the systems in which it is used. Nvidia’s Hopper-based H100 costs between $25,000 and $40,000 per chip, with entire systems costing up to $200,000, according to analyst estimates.
Nvidia will also sell B200 graphics processors as part of a complete system that takes up an entire rack of servers.
NIM
Nvidia also announced that it will add a new product called NIM to its Nvidia enterprise software subscription.
NIM makes it easier to use older Nvidia GPUs for inferencing or running AI software and will allow companies to continue using the hundreds of millions of Nvidia GPUs they already own. Inference requires less computational power than initially training a new AI model. NIM allows companies that want to run their own AI models, rather than purchasing access to AI results as a service from companies like OpenAI.
The strategy is to get customers who buy Nvidia-based servers to sign up for Nvidia enterprise, which costs $4,500 per GPU per year for a license.
Nvidia will work with AI companies like Microsoft or Hugging Face to ensure their AI models are optimized to run on all compatible Nvidia chips. Then, using a NIM, developers can efficiently run the model on their own servers or cloud-based Nvidia servers without a lengthy setup process.
“In my code, where I was calling OpenAI, I’m going to replace a line of code to point it to this NIM that I got from Nvidia,” Das said.
Nvidia says the software will also help the AI run on GPU-equipped laptops, rather than on servers in the cloud.