AMD's Advancing AI event featured a jarring statistic from AMP CEO AJ: current AI infrastructure, the very foundation of the modern tech race, is operating at shocking levels of waste. While the headlines scream about ever-larger models and infinite scaling, the reality on the ground is stark. Less than 20% of data center net flops are being used, and within individual chips, model flop utilization (MFU) is a dismal 15%.

That means a staggering 85% of your expensive AI compute sits idle, waiting for some other process—storage, memory, networking—to catch up. AJ doesn't just call this inefficient; he labels it a "national security crisis." He argues this massive underutilization is what's truly holding the US back from staying at the frontier of AI, especially when compared to rivals like China and their industrial scaling capabilities.

For AJ, the obsession with raw token counts misses the point. The real game is "output maxing," which he defines as "what's the output divided by the unit of input." This means focusing on what your AI actually accomplishes for every dollar, watt, or chip hour, instead of just throwing more hardware at the problem. The core issue? Deeply flawed scheduling and synchronization bottlenecks. AJ highlights that up to "30 40% flops in just bad scheduling. The nodes are just straight up not allocated" or are simply “not getting work at the right time.” If you are buying, building, or even just renting AI compute, this 85% waste is your hidden enemy.

Key Takeaways

  • Current AI infrastructure is shockingly inefficient: data centers hit less than 20% net flop utilization, and individual chips manage only 15% Model Flop Utilization (MFU).
  • This 85% waste isn't just poor economics; AMP CEO AJ calls it a "national security crisis," keeping the US from the AI frontier.
  • Forget pure token counts. The true metric for AI progress is "output maxing": the output achieved divided by the unit of input used.
  • Massive inefficiencies stem from bad data center scheduling and chip bottlenecks, where the silicon waits for memory, storage, or network processes.
  • AMP's Output Maxing Strategy offers a framework to systematically identify and eliminate these hidden pockets of waste in your AI operations.

The AMP's Output Maxing Strategy

  • Definition: Output maxing is defined as 'what's the output divided by the unit of input.'
  • Addressing Data Center Scheduling Waste: Identify and eliminate the 'about 30 40% flops in just bad scheduling. The nodes are just straight up not allocated' or when compute 'is there, but it's just not getting work at the right time.'
  • Addressing Chip Utilization Waste (MFU): Improve the 'only like 15% of the chip being utilized. That's called MFU model flop utilization cuz the chip is waiting around for some other process to complete: storage, memory, networking or some other chip to hand it off.'

When This Works (and When It Doesn't)

This strategy is crucial for measuring businesses or model capabilities through efficiency, especially in the AI scaling era. It is necessary for countries like the United States to stay competitive at the frontier when pure scaling is not an option due to supply chain backlogs and limited resources. Practically, this framework shines when compute resources are a bottleneck—either expensive, scarce, or both. It's particularly useful for founders running production AI services, training large models, or managing substantial cloud AI spend, where every percentage point of efficiency translates directly to cost savings or faster iteration. The strategy requires a clear definition of "output" specific to your use case. It might be less immediately applicable for early research where raw experimentation and quick hacks outweigh deep optimization, but even then, understanding its principles can prevent costly habits from forming.

What to Do With This

If you're a founder running an AI service or training models, apply this today. First, define your key "output." For an inference service, that might be successful requests served per second; for training, it could be valid training steps completed per hour. Next, audit your actual compute utilization. Use tools like nvidia-smi for individual GPUs, or check your cloud provider's resource utilization dashboards (e.g., AWS CloudWatch, GCP Monitoring) to see if your GPUs or nodes are sitting idle, despite being allocated. If you find utilization hovering in the 15-20% range, you have a massive opportunity. Finally, use profiling tools (like PyTorch profiler or NVIDIA Nsight Systems) to pinpoint MFU bottlenecks—are your chips waiting for data? Are network transfers slow? Address these specific scheduling and I/O issues, then re-measure your output per input. Repeat this cycle relentlessly.