Tesla Dojo Timeline | TechCrunch

Elon Musk doesn’t want Tesla to be just a carmaker. He wants Tesla to be an AI company that figures out how to make cars drive themselves.

Crucial to this mission is Dojo, Tesla’s purpose-built supercomputer designed to train Full Self-Driving (FSD) neural networks. FSD isn’t fully autonomous; it can perform some automated driving tasks, but it still requires an attentive human behind the wheel. However, Tesla believes that with more data, more processing power, and more training, it can cross the threshold from near-autonomous to fully autonomous driving.

This is where Dojo comes in.

Musk has been teasing Dojo for a while, but the CEO has been ramping up discussions about the supercomputer throughout 2024. Dojo’s significance to Tesla could be existential — with electric vehicle sales declining, investors want assurance that Tesla can achieve autonomy. Below is a timeline of Dojo’s mentions and promises.

2019

First mentions of Dojo

April 22nd – During Tesla’s Autonomy Day, the automaker had its AI team on stage to talk about Autopilot and fully autonomous driving, and the AI ​​that powers them. The company shared information about Tesla’s custom chips, which are designed specifically for neural networks and autonomous cars.

During the event, Musk previews Dojo, revealing it is a supercomputer for AI training. He also notes that all Tesla cars produced at the time would have all the hardware necessary for full autonomous driving and would only need a software update.

2020

Musk kicks off Dojo tour

February 2nd – Musk says Tesla will soon have more than 1 million connected vehicles worldwide, equipped with sensors and computing power for fully autonomous driving — and he touts the Dojo’s capabilities.

“Dojo, our training supercomputer, will be able to process massive amounts of video data and efficiently handle hyperspatial arrays with massive numbers of parameters, large amounts of memory, and ultrahigh core-to-core bandwidth. More on that later.”

August 14th – Musk reiterates Tesla’s plan to develop a neural network training computer called Dojo “that will process really massive amounts of video data,” calling it a “beast.” He also says the first version of Dojo is “about a year away,” which would put its launch sometime around August 2021.

December 31 — Elon says Dojo isn’t needed, but it will make autonomous driving better. “It’s not enough to be safer than human drivers, Autopilot ultimately needs to be more than 10 times safer than human drivers.”

2021

Tesla Officially Announces Dojo

August 19th – The automaker is officially announcing Dojo at Tesla’s first AI Day, an event designed to attract engineers to Tesla’s AI team. Tesla is also introducing its D1 chip, which the automaker says will be used — alongside Nvidia’s GPUs — to power the Dojo supercomputer. Tesla notes that its AI cluster will house 3,000 D1 chips.

October 12th – Tesla releases AND Dojo Technology white paper, “A Guide to Configurable Floating-Point Formats and Tesla Arithmetic.” The white paper presents a technical standard for a new type of binary floating-point arithmetic that is used in deep learning neural networks and can be implemented “entirely in software, entirely in hardware, or in any combination of software and hardware.”

2022

Tesla reveals progress at Dojo

August 12th – Musk says Tesla will “introduce Dojo. You won’t need to buy as many incremental GPUs next year.”

September 30th – During Tesla’s second AI Day, the company reveals that it has installed its first Dojo cabinet, testing a 2.2 megawatt test load. Tesla says it was building one tile per day (which consists of 25 D1 chips). Tesla demonstrates Dojo on stage, running a Stable Diffusion model to create an AI-generated image of a “Cybertruck on Mars.”

Importantly, the company has set completion of the full Exapod cluster in the first quarter of 2023. It plans to build a total of seven Exapods in Palo Alto.

2023

Long Shot Bet‘

April 19th – During Tesla’s first-quarter earnings call, Musk told investors that Dojo “has the potential to improve training costs by orders of magnitude” and also “has the potential to become a saleable service that we could offer to other companies in the same way that Amazon Web Services offers web services.”

Musk also notes that he would “treat Dojo as a low probability bet” but “a bet worth taking.”

June 21st — The Tesla AI X account says the company’s neural networks are already in customer vehicles. The thread includes a chart with a timeline of Tesla’s current and projected computing power, which puts the start of Dojo production in July 2023, although it’s unclear whether that refers to the D1 chips or the supercomputer itself. Musk says that same day that Dojo was already online and running tasks in Tesla’s data centers.

The company also forecasts that by February 2024, Tesla’s computing power will be among the top five computing power in the world (there is no indication that this plan will be successful), and that by October 2024, Tesla will reach 100 exaflops.

July 19th – Tesla notes in its second-quarter earnings report that it has begun production of the Dojo. Musk also says Tesla plans to spend more than $1 billion on the Dojo by 2024.

September 6 – Musk writes on X that Tesla is limited by AI training computations, but that Nvidia and Dojo will fix that. He says that managing data from the roughly 160 billion video frames Tesla receives from its cars per day is incredibly difficult.

2024

Scaling Plans

January 24th – During Tesla’s fourth-quarter and full-year earnings call, Musk again acknowledged that Dojo is a high-risk, high-reward project. He also says that Tesla has been on a “dual path of Nvidia and Dojo,” that “Dojo is working” and “doing the training.” He notes that Tesla is scaling it and has “plans for Dojo 1.5, Dojo 2, Dojo 3, and so on.”

January 26 – Tesla announced plans to spend $500 million to build a Dojo supercomputer in Buffalo. Musk then downplayed the investment somewhat, posting on X that while $500 million is a lot, it “only matches Nvidia’s $10,000 H100 system. Tesla will spend more on Nvidia hardware this year. The stakes for being competitive in AI are now at least a few billion dollars a year.”

April 30th – During TSMC’s North American Technology Symposium, the company said its next-generation Dojo training board — the D2, which places an entire Dojo tile on a single silicon wafer rather than combining 25 chips to create a single tile — is now in production, according to IEEE Spectrum.

May 20 – Musk notes that the back end of the Giga Texas factory expansion will involve building a “super-dense, water-cooled supercomputer cluster.”

June 4 – CNBC report reveals Musk diverted thousands of Nvidia chips reserved for Tesla to X and xAI. After initially declaring the report false, Musk posts on X that Tesla didn’t have the space to ship the Nvidia chips to power them up, due to ongoing construction on the southern extension of Giga Texas, “so they’d just sit in storage.” He noted that the extension would “hold 50,000 H100s for FSD training.”

He too posts:

“Of the roughly $10 billion in AI spending that I said Tesla will do this year, about half is internal, primarily the Tesla-designed AI inference computer and the sensors that are in all of our cars, plus Dojo. In the case of building AI training superclusters, Nvidia hardware is about 2/3 of the cost. My current best guess is that Tesla will buy between $3 billion and $4 billion in Nvidia this year.”

July 1st – Musk reveals at X that Tesla’s current vehicles may not have the right hardware for the company’s next-generation AI model. He says the roughly 5x increase in parameters for next-generation AI “is very difficult to achieve without upgrading the vehicle’s inference computer.”

Nvidia Supply Challenges

July 23 – During Tesla’s second-quarter earnings press conference, Musk said demand for Nvidia hardware is “so strong that GPUs are often hard to come by.”

“I think it requires us to put a lot more effort into Dojo to make sure we have the training capabilities that we need,” Musk says. “And we see a path to being competitive with Nvidia with Dojo.”

A chart on Tesla’s Investor Deck predicts that Tesla’s AI training capacity will grow to about 90,000 H100-equivalent GPUs by the end of 2024, up from about 40,000 in June. Later that day at X, Musk posts that Dojo 1 will have “about 8k H100-equivalent online training by the end of the year.” He also posts photos of the supercomputer, which appears to use the same stainless steel refrigerator as Tesla’s Cybertrucks.

XXX

July 30 – According to Musk, the AI5 is still about 18 months away from starting mass production. answer to a post by someone claiming to have started a club of “Tesla HW4/AI4 owners angry at being left behind when AI5 comes out.”

August 3rd – Musk posts on X that he has reviewed the “Tesla supercomputing cluster at Giga Texas (aka Cortex).” He notes that it will be built with about 100,000 Nvidia H100/H200 GPUs with “massive storage for FSD and Optimus video training.”