Key Takeaways

  • Sam Altman argues market forces cannot resolve the compute crunch because memory manufacturers fear the boom-and-bust cycles that previously wiped out their margins.
  • Memory makers historically spent $20 billion in capital expenditures only to generate $19 billion in revenue, creating structural hesitation to build new fabs today.
  • Local device inference creates massive allocative waste by locking scarce DRAM into consumer laptops running at 5% utilization instead of 100% cloud saturation.
  • The five-year total cost of ownership for data centers sits near $70 billion per gigawatt, a figure Altman wants to drive down to $20 billion.
  • OpenAI is expanding into proprietary chip design and robotics specifically to automate the physical construction of power plants, fabs, and data centers.

The Memory Market Is Traumatized by Its Own History

Technologists assume that high prices automatically summon new supply. If compute demand spikes, chipmakers should pour concrete and build fabs. But Altman told John Collison that market coordination has stalled out in the semiconductor supply chain.

“I think the invisible hand is failing right now,” Altman noted. “The invisible hand... It has a checkered track record at best. But, I think it's not quite working here.”

The breakdown starts with memory. AI training and inference require massive pools of high-bandwidth memory, yet DRAM suppliers refuse to build ahead of demand. They remember what happened the last time they tried. Altman pointed out the brutal math of previous memory cycles: “For a very long time, if you were a producer of DRAM, you would invest $20 billion of CapEx to make $19 billion. And you would go through these booms and busts.”

Because suppliers spent decades suffering through catastrophic gluts, boardroom incentives favor undersupplying the market over risking another multi-billion-dollar write-down. The market cannot clear fast enough when the suppliers are conditioned by survival to move slowly.

Local Inference Hoards Scarce Silicon

The second market distortion comes from users wanting models to run directly on their phones and laptops. While consumer tech companies promote on-device inference as a privacy win, Altman views it as a misallocation of global resources.

“From a global allocation perspective, actually, it's quite terrible to have everybody have a bunch of the world's limited supply of DRAM sitting in their own computer. And, being used... 5% of the time,” Altman argued.

Every gigabyte of high-performance memory soldered into an idle consumer laptop is a gigabyte kept away from shared cloud clusters running at constant utilization. When hardware supply is strictly constrained by physical fab capacity, distributed computing starves centralized workloads that generate vastly more economic output per watt and per chip.

Self-Replicating Infrastructure Is the Only Scale Solution

To break the ceiling, the cost structure of building AI infrastructure must drop by two-thirds. Altman estimates that the five-year total cost of ownership for compute capacity is currently hovering near $70 billion per gigawatt: “I think if that number were... 20 instead of 70, we should be much happier.”

Private markets cannot produce that cost reduction by simply writing bigger checks to existing vendors. That economic reality explains why OpenAI is vertically integrating into custom chip design and physical automation. Altman argues that the real frontier is not software that writes software, but software that builds hardware.

“People talk a lot about RSI as an AI that does AI research,” Altman said. “There's very little, relatively little talk about self-replicating data centers. This idea that you can use AI to control robots, and run fabs, and build turbines or solar panels, and build data centers.”

Until software directs physical machines to erect substations and operate fabs, compute scaling will remain chained to human supply chains that move too slowly to satisfy the demand.

What to Do With This

Audit your product roadmap and strip out plans that rely on on-device model execution for heavy workflows. Anchor your architecture to high-utilization cloud endpoints where memory is shared, and negotiate multi-year reserved cloud capacity today before fab bottlenecks tighten allocation further.