AI chip startup Groq Inference Chip raises $640M – High Performance Computing News Analysis

Amid the gloom of Wall Street’s tech crisis, an AI inference chip startup Groq today announced that it has raised $640 million in a Series D funding round at a valuation of $2.8 billion.

Groq also announced that Stuart Pann, a former senior executive at HP and Intel, has joined the leadership team as COO. And Groq announced the addition of its newest technical advisor, Yann LeCun, vice president and chief AI scientist at Meta.

The round was led by funds and accounts managed by BlackRock Private Equity Partners with participation from existing and new investors, including Neuberger Berman, Type One Ventures, and strategic investors including Cisco Investments, Global Brain’s KDDI Open Innovation Fund III, and Samsung Catalyst Fund. Groq AI’s unique, vertically integrated inference platform has generated a surge in demand from developers seeking exceptional speed.

“We are extremely impressed with Groq’s groundbreaking computing architecture and software-first approach. Groq’s record-breaking speed and near-instantaneous generative AI inference performance lead the market,” said Marco Chisari, head of Samsung Semiconductor Innovation Center and executive vice president of Samsung Electronics.

“You can’t power AI without inference computation,” said Jonathan Ross, CEO and founder of Groq. “We’re committed to making resources available so anyone can build cutting-edge AI products, not just the biggest tech companies. This funding will allow us to deploy over 100,000 additional LPUs to GroqCloud. We’ve solved the AI ​​model training, now it’s time to deploy those models for the world to use. With twice the funding secured, we now plan to significantly increase our talent density. We’re a team that’s enabling hundreds of thousands of developers to build on open models, and we’re hiring.”

Groq announced that the company has grown to over 360,000 developers who are building their businesses GroqCloudcreating AI applications based on openly available models such as Llama 3.1 with MetaWhisper Large V3 from OpenAI, Gemma from Google and Mixtral from Mistral.

Groq said it will use the funds raised to expand its token-as-a-service (TaaS) offering and add new models and features to GroqCloud.

As Gen AI applications move from training to deployment, developers and enterprises need an inference strategy that meets user and market demands for speed, according to Groq. To meet developer and enterprise demand, the company will deploy more than 108,000 Groq AI LPUs manufactured by GlobalFoundries by the end of Q1 2025, which will be the largest AI inference compute deployment of any non-hyperscaler.

Mohsen Moazami, President International at Groq and former Emerging Markets Leader at Cisco, is leading commercial efforts with enterprises and partners, including Aramco Digital and Earth Wind & Power, to build AI computing centers around the world, allowing developers to access Groq technology regardless of their location.

“Aramco Digital is partnering with Groq to build one of the largest AI Inference-as-a-Service computing infrastructures in the MENA region,” said Tareq Amin, CEO of Aramco Digital. “Our close collaboration with Groq is transformative for both domestic and global AI demand.”

Groq LPU’s AI inference technology was designed with software in mind to address the unique characteristics and needs of AI. This approach gave Groq the advantage of quickly delivering new models to developers. The investment will enable Groq to accelerate the next two generations of LPUs, according to the company.

Morgan Stanley & Co. LLC acted as exclusive placement agent for Groq in this transaction.