Google Just Released Gemma 2 — The Power of GPT-3.5 on Your Phone Is Now Available

Google DeepMind has just released a completely new model of artificial intelligence, Gemma 2 2BThis is a 2 billion parameter model, small enough to fit in a smartphone, and yet offers performance at the GPT 3.5 level.

The launch of this device follows the June announcement of the Gemma 2 9B and 27B models, which are Google’s response to competition in the Llama AI sector, in particular Meta and its Llama 3.1 family.

The new Gemma 2B is further evidence of a deliberate move to smaller, lighter models that can run on a wider range of devices, not just powerful PCs.

Even though the new LLM is a fraction of the physical size of previous models such as OpenAI’s GPT-3.5 and GPT-4, its benchmark performance reflects the vast improvements in training and batching it now offers.

Donut

(Image source: Google DeepMind)

According to Google, the new model offers “best-in-class performance for its size, outperforming other open models in its category,” and the LYMSYS charts certainly show off some impressive stats.

The model apparently achieves its performance improvement “by learning from larger models through distillation.” The technique was introduced in a 2015 paper by Geoffrey Hinton and uses an advanced method of training students and teachers that allows for the creation of smaller, more compressed models.

So for most people, cloud-based AI will likely continue to dominate, at least in the short term. But inevitably, as the technology matures, on-device AI will get a bigger slice of the chatbot pie.

The market for mobile and edge-based Al for use on small portable and static devices such as sensors is expected to grow exponentially in the coming years. This is particularly true for multimodal AI applications that will use vision and sound to create a rich interactive universe for the user. Hence the race to perfect smaller, more efficient systems as quickly as possible.

However, despite all the talk about running this type of AI locally on smartphones (so-called “on-device”), the reality is that there are still very few mobile devices that can meet the processing demands of current LLM systems.

Modern iPhones running iOS 14 and newer Android phones using Snapdragon processors like the Galaxy S23 Ultra are among those that can do this. But even then, performance is modest compared to desktop systems, and Apple’s Apple Intelligence, which is mostly on the device, requires at least an iPhone 15.

So for most people, cloud-based AI will likely continue to dominate, at least in the short term. But inevitably, as the technology matures, on-device AI will take a bigger slice of the chatbot pie. Especially as users start to focus more and more on privacy, security, and reliability.

Interested readers can check out the new Gemma 2 2B model at Google AI Studio playground and is available to run on your device via Ollama.