Nvidia is the undisputed king of AI chips. But thanks to the AI it helped build, the champ could soon face growing competition.
Modern AI runs on Nvidia designs, a dynamic that has propelled the company to a market cap of well over $4 trillion. Each new generation of Nvidia chip allows companies to train more powerful AI models using hundreds or thousands of processors networked together inside vast data centers. One reason for Nvidia’s success is that it provides software to help program each new generation of chip. That may soon not be such a differentiated skill.
A startup called Wafer is training AI models to do one of the most difficult and important jobs in AI—optimizing code so that it runs as efficiently as possible on a particular silicon chip.
Emilio Andere, cofounder and CEO of Wafer, says the company performs reinforcement learning on open source models to teach them to write kernel code, or software that interacts directly with hardware in an operating system. Andere says Wafer also adds “agentic harnesses” to existing coding models like Anthropic’s Claude and OpenAI’s GPT to soup up their ability to write code that runs directly on chips.
Many prominent tech companies now have their own chips. Apple and others have for years used custom silicon to improve the performance and the efficiency of software running on laptops, tablets, and smartphones. At the other end of the scale, companies like Google and Amazon mint their own silicon to improve the performance of their cloud-computing platforms. Meta recently said it would deploy 1 gigawatt of compute capacity with a new chip developed with Broadcom. Deploying custom silicon also involves writing a lot of code so that it runs smoothly and efficiently on the new processor.
Wafer is working with companies including AMD and Amazon to help optimize software to run efficiently on their hardware. The startup has so far raised $4 million in seed funding from Google’s Jeff Dean, Wojciech Zaremba of OpenAI, and others.
Andere believes that his company’s AI-led approach has the potential to challenge Nvidia’s dominance. A number of high-end chips now offer similar raw floating point performance—a key industry benchmark of a chip’s ability to perform simple calculations—to Nvidia’s best silicon.
“The best AMD hardware, the best [Amazon] Trainium hardware, the best [Google] TPUs, give you the same theoretical flops to Nvidia GPUs,” Andere told me recently. “We want to maximize intelligence per watt.”
Performance engineers with the skill needed to optimize code to run reliably and efficiently on these chips are expensive and in high demand, Andere says, while Nvidia’s software ecosystem makes it easier to write and maintain code for its chips. That makes it hard for even the biggest tech companies to go it alone.
When Anthropic partnered with Amazon to build its AI models on Trainium, for instance, it had to rewrite its model’s code from scratch to make it run as efficiently as possible on the hardware, Andere says.
Of course, Anthropic’s Claude is now one of many AI models that are now superhuman at writing code. So Andere reckons it may not be long before AI starts consuming Nvidia software advantage.
“The moat lives in the programmability of the chip,” Andere says in reference to the libraries and software tools that make it easier to optimize code for Nvidia hardware. “I think it’s time to start rethinking whether that’s actually a strong moat.”
Besides making it easier to optimize code for different silicon, AI may soon make it easier to design chips themselves. Ricursive Intelligence, a startup founded by two ex-Google engineers, Azalia Mirhoseini and Anna Goldie, is developing new ways to design computer chips with artificial intelligence. If its technology takes off, a lot more companies could branch into chip design, creating custom silicon that runs their software more efficiently.
No Comment! Be the first one.