AI chip startup Groq raises $640 million to challenge Nvidia

Groq, a startup that develops chips to run generative AI models faster than conventional processors, said Monday it has raised $640 million in a new funding round led by Blackrock. Neuberger Berman, Type One Ventures, Cisco, KDDI and Samsung Catalyst Fund also participated.

The tranche, which brings the total amount raised by Groq to over $1 billion and values ​​the company at $2.8 billion, is a major win for Groq, which was reportedly originally looking to raise $300 million at a slightly lower ($2.5 billion) valuation. This is more than double Groq’s previous valuation (~$1 billion) in April 2021, when the company raised $300 million in a round led by Tiger Global Management and D1 Capital Partners.

Meta’s chief AI scientist Yann LeCun will serve as a technical advisor to Groq, and Stuart Pann, former head of Intel’s foundry business and former CIO at HP, will join the startup as chief operating officer, Groq also announced today. LeCun’s appointment is somewhat unexpected given Meta’s investment in its own AI chips — but it undoubtedly gives Groq a powerful ally in the cutthroat space.

Groq, which emerged from stealth in 2016, is building what it calls a language processing unit (LPU) inference engine. The company says its LPUs can run existing generative AI models similar in architecture to OpenAI’s ChatGPT and GPT-4o with 10x the speed and one-tenth the energy.

Groq CEO Jonathan Ross is famous for helping invent the tensor processing unit (TPU), Google’s custom AI accelerator chip used to train and run models. Ross teamed up with Douglas Wightman, an entrepreneur and former engineer at Google parent Alphabet’s X moonshot lab, to co-found Groq nearly a decade ago.

Groq provides an LPU-powered developer platform called GroqCloud, which offers “open” models like Meta’s Llama 3.1 family, Google’s Gemma, OpenAI’s Whisper, and Mistral’s Mixtral, as well as an API that lets customers use its chips on cloud instances. (Groq also hosts an AI-powered chatbot playground, GroqChat, which it launched late last year.) As of July, GroqCloud had more than 356,000 developers; Groq says a portion of the proceeds from the round will be used to scale capacity and add new models and features.

“Many of these developers work at large enterprises,” Stuart Pann, Groq’s chief operating officer, told TechCrunch. “We estimate that more than 75% of the Fortune 100 are represented.”

As the generative AI boom continues, Groq faces increasing competition from both new AI chip companies and AI hardware powerhouse Nvidia.

Nvidia is estimated to control between 70% and 95% of the market for AI chips used to train and deploy generative AI models, and the company is taking aggressive steps to maintain its dominant position.

Nvidia has committed to releasing a new AI chip architecture every year, rather than every two years as it has in the past. And it is reportedly creating a new business unit focused on designing custom chips for cloud computing companies and others, including AI hardware.

In addition to Nvidia, Groq competes with Amazon, Google, and Microsoft, which all offer — or will soon offer — custom chips for AI workloads in the cloud. Amazon has its Trainium, Inferentia, and Graviton processors available through AWS; Google Cloud customers can use the aforementioned TPUs, and eventually Google’s Axion chip; and Microsoft recently launched Azure preview instances for its Cobalt 100 processor, with Maia 100 AI Accelerator instances expected to launch in the next few months.

Groq could also be eyeing Arm, Intel, AMD and a growing number of startups as rivals in the AI ​​chip market, which could reach $400 billion in annual sales in the next five years, according to some analysts. Arm and AMD in particular have thriving AI chip businesses, thanks to rising capital spending by cloud providers to meet the demand for processing power for generative AI.

D-Matrix raised $110 million late last year to commercialize what it describes as a first-of-its-kind computing platform for inference. In June, Etched came out of stealth with $120 million for a processor purpose-built to accelerate the dominant generative AI model architecture, Transformer. SoftBank’s Masayoshi Son is reportedly looking to raise $100 billion for a chip venture to compete with Nvidia. And OpenAI is reportedly in talks with investment firms to launch an AI chip manufacturing initiative.

To carve out a niche for itself, Groq is investing heavily in corporate and government activities.

In March, Groq acquired Definitive Intelligence, a Palo Alto-based company offering a range of business-focused AI solutions, to create a new business unit called Groq Systems. Groq Systems’ scope includes serving organizations, including U.S. government agencies and sovereign nations, that want to add Groq chips to existing data centers or build new data centers using Groq processors.

Groq recently partnered with Carahsoft, a government IT contractor, to sell its solutions to public sector customers through Carahsoft’s channel partners. The startup signed a letter of intent to install tens of thousands of LPUs in a Norwegian data center owned by European company Earth Wind & Power.

Groq is also working with Saudi consulting firm Aramco Digital to install LPUs in future data centers in the Middle East.

At the same time it’s building relationships with customers, Mountain View, California-based Groq is moving toward the next generation of its chip. Last August, the company announced it had signed a contract with Samsung’s foundry to produce 4nm LPUs, which are expected to offer performance and efficiency gains over Groq’s first-generation 13nm chips.

Groq says it plans to deploy over 108,000 LPUs by the end of Q1 2025.