Key Takeaways

  • Google's Gemma 4, an on-device model, already matches the state-of-the-art capabilities from 1 to 1.5 years ago for local functions like agentic tasks and conversational AI.
  • While small models excel at executing instructions and handling privacy-sensitive tasks on a device, they face clear limits in deep knowledge and factuality, which still demand larger, cloud-based models.
  • Omar Sanseviero of Google DeepMind argues that models like Gemma and Gemini do not cannibalize each other; instead, they serve fundamentally distinct purposes in Google's AI strategy.
  • Founders should think about their AI use case not just by a model's general capability, but by its core requirement: instant, local execution and privacy, or deep, factual reasoning and complex problem-solving.

Gemma's Local Power: Where On-Device AI Shines

For any founder building products that demand speed, personalization, or strict privacy, the rise of powerful on-device AI is a game-changer. Omar Sanseviero of Google DeepMind makes it clear that their Gemma 4 model already brings substantial capabilities directly to devices. “I would say Gemma 4 is matching state-of-the-art from 1 1 and 1/2 years ago for most things,” Sanseviero explained. This isn't about general intelligence, but about specific, actionable functions. He points to areas where Gemma excels: “With local models or models that you can run in your own hardware, you can get capabilities, so you can get agentic capabilities, function calling, system instructions, like conversational and that kind of stuff.”

Imagine an intelligent agent on a user's phone that can manage schedules, draft emails, or control device settings, all without sending data to the cloud. This is the sweet spot for models like Gemma. The ability to perform complex, multi-step agentic tasks locally opens up a wave of product experiences that are faster, more reliable offline, and inherently more private. It's about bringing immediate, intelligent action to the user's hand.

Gemini's Enduring Edge: The Knowledge Frontier

Despite the rapid advancements in on-device AI, Sanseviero is clear that larger, cloud-based models like Google's Gemini retain a critical, unchallenged advantage: knowledge. “Knowledge is much trickier,” he says. “For knowledge, you do need a larger model, right? That's why if you compare Gemini to Gemma, Gemini is much better knowledge and of the world.” This distinction is vital. While a local model can execute tasks, it simply cannot hold the vast, intricate web of facts and contextual information that a massive cloud model can.

Founders chasing products that require deep factual accuracy, extensive research, complex reasoning over broad datasets, or generating highly nuanced content need to look to cloud-based solutions. Sanseviero emphasizes, “If you want like flagship capabilities, this super complex, long-running task you would do with Gemini if you need factuality and so on.” He adds a stark reality: even highly capable future local models “will not know like who was the president of X country 25… very niche knowledge probably the models will not have.” For applications demanding a truly comprehensive understanding of the world, bigger is still better.

Separate Paths: Why Cannibalization Is A Myth

Swyx, the host, put the central question directly to Sanseviero: “Do you see a future where, you know, small models get good enough? Like, does it cannibalize? It's an interesting position. Like, you have big Gemini, you have Gemma, both get exponentially better over time.” Sanseviero's answer cuts through the industry chatter: he doesn't see cannibalization. “I wouldn't say it cannibalizes. Still like two very different things.”

This isn't just Google's internal messaging; it's a strategic recognition of distinct market needs. Gemma empowers new, private, on-device agentic experiences. Gemini continues to drive flagship, knowledge-intensive, and complex cloud-based AI services. The vision isn't about one replacing the other, but about powerful models running directly on phones, enabling rich product experiences, while cloud giants handle the heavy lifting of global knowledge and complex reasoning. They complement, rather than compete directly for the same use cases.

What to Do With This

If you're building an AI product, stop asking if a model is "good enough" in general. Instead, identify your core value proposition: Is it rooted in on-device execution, privacy, and speed (e.g., a smart local assistant, automated device control)? Or is it dependent on deep factual knowledge, complex reasoning, and broad information retrieval (e.g., a research tool, advanced content generation, financial analysis)? Test your hypothesis immediately. Prototype with a small, locally runnable model (or its open-source equivalent) for agentic tasks, and separately with a large cloud API for knowledge-intensive ones. The performance gaps will quickly tell you which path your product's fundamental needs fall into, saving you weeks of misdirected effort.