Key Takeaways

  • General fine-tuning for broad conversational changes in large language models is rapidly becoming unnecessary, with models like Google DeepMind's Gemma 4 performing exceptionally well out-of-the-box.
  • Google DeepMind observed that out of 50 to 60 partners, many initially aimed to fine-tune even a 27B model for tasks like vision, only to discover the base model's capabilities made the effort redundant.
  • Behavioral shifts and general model customization that once required deep fine-tuning are now largely achievable through sophisticated prompting techniques, shifting the skill set required from engineers.
  • The future of fine-tuning for specific tasks will increasingly be managed by non-coders using agentic tools and "skills" platforms (like those from Hugging Face) that automate experimental setups through natural language prompts.
  • Targeted fine-tuning remains critical for niche domains such as finance or healthcare, where unique, proprietary datasets are essential to instill specialized knowledge that base models lack.

The Quiet Demise of General Fine-Tuning

For a period around 2023 and 2024, the buzz around LLM fine-tuning was everywhere. Communities sprang up, and everyone seemed to be tweaking models for every conceivable use case. But Omar Sanseviero from Google DeepMind brings a stark reality check: that era of widespread, general fine-tuning is rapidly ending. Why? Because the models themselves are simply getting too good.

Sanseviero notes a significant shift. “Models are getting very good out of the box,” he explains, effectively killing the need for many to customize general conversational behaviors. He pointed to a telling anecdote from Google DeepMind's own partnerships. “We had 50 to 60 partners and some of them were like oh yeah we're going to try and fine-tune the 27B model for this vision task and then they were like oh actually the model works too well out of the box we don't need to fine-tune it.” This isn't a minor tweak; it's a fundamental change in how teams interact with these powerful systems. The base models, like Gemma 4 with its novel E2B architecture, are doing the heavy lifting straight away, making bespoke behavioral fine-tuning an inefficient use of resources.

Prompting Eats Code, Agents Handle the Rest

The implications for builders are clear: if you're trying to adjust a model's tone, style, or general conversational patterns, you're likely barking up the wrong tree by diving into complex fine-tuning. Sanseviero says, “as general conversational like just changing how the model behaves you can do most of that via prompting nowadays.” This means the leverage point for customization has moved from data scientists meticulously preparing datasets and training runs, to prompt engineers crafting precise instructions.

But what about those truly specific use cases? For domains like finance or healthcare, where models need to understand highly specialized jargon or regulatory compliance, fine-tuning still has a place. This is where models learn from unique, proprietary data they haven't seen. However, even this specialized fine-tuning is evolving beyond manual code. Sanseviero predicts the next wave of fine-tuners won't be coding at all. He envisions a future where “most people will be fine-tuning with a couple skills, right? Like Hugging Face has a skills, like all of these libraries have skills. They will just prompt the agent to kick off like some experiments and see what works, what doesn't work.”

This paints a picture where the deep architectural research remains a specialized coding task, but the application of fine-tuning becomes democratized through agentic tools. Think of it: a founder could simply describe the desired outcome, and an AI agent would manage the data preparation, training, and evaluation, automating away the tedious, experimental process that once required custom Python scripts.

What to Do With This

Immediately re-evaluate your LLM strategy. Before committing engineering time to fine-tuning for general behavioral changes, push base models hard with advanced prompt engineering; you'll likely find the performance gains are minimal for the effort. For domain-specific knowledge, shift your focus from building custom fine-tuning pipelines to exploring upcoming agentic fine-tuning tools and platforms that will allow non-coders to manage specialized model customization through natural language commands.