Key Takeaways

  • AI models are now getting good at autonomously optimizing their own underlying infrastructure. Baseten’s GLM 5.2 model proves this by rewriting its own GPU kernels.
  • This creates a continuous improvement loop: live inference data directly feeds into the model’s ability to refine its performance, not just its outputs.
  • Philip Kiely predicts that many leading agent builders will have these self-optimizing loops in production within a couple of years, making agents dynamically better.
  • The shift is towards both “training for inference” (making models fast) and “inference for training” (using live runs to make models smarter).

The Method: When Your AI Learns to Tune Itself

Imagine an AI model that doesn't just process information, but actively scrutinizes its own performance bottlenecks and then rewrites its own code to fix them. Ali Taha from Baseten shared a concrete example from their internal operations that sounds like something out of a sci-fi novel. Their GLM 5.2 model, deployed as a live endpoint, is doing exactly this.

Here’s how it works: The GLM 5.2 instance runs a forward pass, much like any other model doing its job. But then, it generates a profile trace. This trace is a detailed map of where the model spends its computational energy, highlighting any slow spots. Taha explained, “It will do a forward pass on the JLM 52 instance of the you know, the node. And then it will get the profile trace and it will analyze it and it will find the kernels that are the bottlenecks in that JLM.”

Once identified, GLM 5.2 doesn't just report the issue to a human. Instead, it proceeds to write new GPU kernels – the low-level code that dictates how the graphics processing unit handles its tasks. It implements these new kernels, then runs another profiling trace to verify the improvements. If all checks out, the optimized image is uploaded, ready for future deployments. Taha stated plainly, “some of the GPU kernels that we run on JLM52 within our inference engine is written by JLM52.” This isn't theoretical; it's already happening.

This isn't just about tweaking hyperparameters; it's about the model optimizing its own foundational infrastructure for speed and efficiency. Philip Kiely, co-host of Latent Space, saw the writing on the wall: “I think within a few months to a couple years like a lot of leading agent builders are going to have these loops like really up and running in production where you are doing influence learning from the influence.”

Where This Breaks Down: The Human Element (For Now)

While GLM 5.2’s self-optimization is jaw-dropping, this method isn't for every founder or every model today. Implementing a system where an AI agent writes and deploys its own low-level GPU kernels requires a highly specialized team, deep expertise in inference engineering, and a model with the specific capabilities of GLM 5.2 – one that excels at code generation and analysis. Most startups aren't building models capable of modifying their own compute architecture. It also demands an incredibly robust, automated deployment pipeline and rigorous safety mechanisms to prevent an AI from accidentally optimizing itself into a corner, or worse, introducing errors. The complexity and potential for unintended consequences mean human oversight and control remain absolutely vital, especially in production environments with high stakes. For general-purpose AI agents, the ability to self-optimize at this low level is still a distant goal.

What to Do With This

Don't wait for your model to write its own GPU kernels. Instead, take the core idea of "inference for training" and apply it to your current systems. This week, identify a critical customer-facing flow or internal process that runs on an AI agent, then design a direct feedback loop. For example, if you use an LLM for customer support responses, instrument it to log user satisfaction scores or follow-up rates. Then, feed those metrics directly back to an automated system that tweaks the model’s prompts, parameters, or even the data it was trained on. Your goal is to build a closed loop where live performance data isn't just monitored, but actively and automatically drives the system's own improvement. Think about making your AI agents not just perform, but evolve in production.