Letta, one of the most anticipated AI startups at UC Berkeley, has just come out of hiding

A startup called Letta has just come out of stealth with technology that helps AI models remember users and conversations. Created at UC Berkeley’s famed startup factory Labs, it also announced $10 million in seed funding led by Felicis’ Astasia Myers, at a post-money valuation of $70 million.

Letta also benefits from the backing of some of the most prominent angel investors in AI, including Jeff Dean of Google, Clem Delangue of Hugging Face, Cristóbal Valenzuela of Runway and Robert Nishihara of Anyscale.

Founded by Berkeley PhD students Sarah Wooders and Charles Packer, it’s a highly anticipated AI startup. That’s because it’s the brainchild of Berkeley’s Sky Computing Lab and is a commercial arm of the popular open-source project MemGPT.

Berkeley’s Sky Computing Lab, led by renowned professor and Databricks co-founder Ion Stoica, is the offspring of RISELab and AMPLab, which spawned companies such as Anyscale, Databricks, and SiFive. In particular, Sky Lab spawned numerous popular open-source large language model (LLM) projects, including Gorilla LLM, vLLM, and the LLM structured language SGLang.

“A lot of projects came out of the lab very quickly, within a year. Just people sitting next to us,” Wooders described. “So it was an amazing time.”

One such project is MemGPT. It is so popular that it went viral even before its release.

“Someone beat us to it,” Packer told TechCrunch. The founders published a whitepaper on Thursday, Oct. 12, 2023, and planned to release a more detailed whitepaper and code on GitHub the following Monday. Some random person found the whitepaper, posted it to Hacker News on Sunday, and “it went viral on Hacker News before we had a chance to properly release the code or publish the whitepaper or, like, start a Twitter thread or something like that,” he said.

The reason for the excitement was that MemGPT alleviates a pernicious problem for LLM: In their native form, models like ChatGPT are stateless, meaning they don’t store historical data in long-term memory. That’s problematic for AI applications that rely on learning from and understanding a user over time—from customer service bots to apps that track health care symptoms. MemGPT manages data and memory so AI agents and chatbots can remember previous users and conversations.

The newspaper post stayed at the top of Hacker News, a popular developer site run by Y Combinator, for 48 hours, Packer said. So he spent the weekend and the next few days answering questions on the site, trying to get the code ready for release. Once the project was live on GitHub, a link to it went viral on Hacker News, again. YouTube interviews and tutorials, Medium posts, 11,000 stars and 1.2K GitHub forks quickly followed.

Myers of VC Felicis also learned about Wooders and Packer while reading about MemGPT and immediately realized the commercial possibilities of the technology.

“I saw that paper when it came out,” she told TechCrunch, and immediately reached out to the founders. “We had an investment theme around AI agent infrastructure, and we appreciated that a really important piece of that was managing data and memory to make these conversational chat bots and AI agents effective.”

The founders continued to virtually drive down Sand Hill Road, on Zoom calls with venture capitalists before choosing the one who first loved them.

In the meantime, Stoica was brokering connections with Dean, Nishihara, and other prominent Silicon Valley angel investors. “A lot of the Berkeley professors, just by virtue of being at Berkeley, are very well-connected,” Packer recalled, describing how easy the angel investor process was. “They have their eye on projects coming out of this lab that are going to be commercialized.”

Competition and the threat from OpenAI o1

While MemGPT is already available and in use, the commercial version of Letta, Letta Cloud, is not yet open for business. As of Monday, Letta is accepting requests from beta users. It will offer a hosted agent service that allows developers to deploy and run stateful agents in the cloud, accessible via REST APIs, a programming interface that can maintain state. Letta Cloud will store the long-term data necessary for this purpose. Letta will also offer developer tools for building AI agents.

With MemGPT, Wooders sees a wide range of applications. “I think the most common use case we see is basically highly personalized, highly engaging chatbots,” he says. But there are also novel applications, such as a “chatbot for cancer patients,” where patients upload their history and then share their current symptoms so the bot can learn and offer guidance over time.

It’s worth noting that MemGPT isn’t the only one working on this. LangChain is probably its best-known competitor and already offers commercial options. Major modelers also offer tools for creating AI agents, such as the OpenAI Assistant API.

And OpenAI’s new o1 model could make the need for state fixing a moot point for users. Because it’s a multi-stage model, it essentially needs to maintain some level of state to “think” and fact-check before responding.

But Wooders, Packer, and Myers see several key differences between what Letta offers and what the 800-pound gorilla of the market, OpenAI, does. Letta says it will work with any AI model, and it expects its users to use a variety of them: OpenAI, Anthropic, Minstrel, their own models. OpenAI’s technology currently works only with itself.

More importantly, Letta uses the open-source MemGPT code and firmly sides with the open-source side of the FOSS vs. black-box LLM debate, arguing that open-source is the better choice for AI application developers.

“We’re positioning ourselves as an open alternative to OpenAI,” Packer says. “I think it’s actually very, very hard to build very good AI applications, especially when you’re after something like hallucination, if you can’t see what’s going on under the hood.”