Key Takeaways

  • The primary ceiling on AI expansion has shifted from GPU allocation and wafer supply to grid capacity, physical land, and substation hookups.
  • Chip operators face a direct trade-off between generation speed and power efficiency, forcing a choice between peak token velocity and tokens delivered per megawatt.
  • China holds a structural lead in AI power delivery due to gigawatt-scale hydroelectric projects, while Western builds battle multi-year permitting disputes and civic resistance.
  • Because data center megawatt capacity is strictly capped, operators must swap out silicon every few years, creating an eviction timeline for existing hardware.
  • Walter Goodwin projects that every single AI chip inside modern data centers will be cleared out within six or seven years to make room for denser architectures.

The Power Interconnect Wall

For two years, founders treated chip supply as the single gatekeeper of intelligence. If you had capital, you waited on GPU allocations from Nvidia and booked cloud credits months in advance. That constraint broke, only to run headfirst into physical physics.

Goodwin tracked this rapid evolution inside modern facilities. As he points out, “it feels like every quarter the thing that is most constraining our ability to run more frontier models at scale in the data center has shifted.” (play from 35:57) The newest constraint is not silicon availability. It is whether a utility company can route high-voltage lines to a building.

Data centers cannot simply draw unlimited current. Substation queues now stretch across years, and land with direct transmission access has dried up. Hardware teams find themselves holding racks of compute that they cannot fully energize without tripping local grids. Raw compute supply has become an energy problem.

The Efficiency Penalty

When power becomes the hard boundary, the engineering metrics inside the cluster flip. Startups often benchmark chips purely on latency: how many milliseconds until the first token arrives, or how fast a model spits out answers. Goodwin highlights that pushing silicon for pure speed destroys energy economy.

“On these chips, you can either be slow and efficient in generating tokens or fast but deeply inefficient: far fewer tokens per second per megawatt of power,” (play from 38:10) Goodwin notes. When an operator runs an accelerator at maximum clock speeds to shave off ten milliseconds of user latency, power consumption spikes nonlinearly. In an unconstrained grid, that waste is just an electricity bill. In a power-capped facility, that waste directly caps total cluster throughput.

China's Megawatt Edge

While Western operators wrestle with local zoning boards and slow utility interconnection studies, other regions move with deliberate speed. Goodwin points out that state-backed infrastructure gives Chinese facilities an immediate physical leg up.

“I think this is one of the places where China has somewhat stolen the march on the West,” (play from 40:22) Goodwin says. “In particular, these mega projects, huge hydro projects, for example, they're ideal for AI compute buildouts.” (play from 40:31) Gigawatt-scale hydropower sites provide massive, baseload clean energy directly adjacent to potential server halls, skipping the decade-long transmission fights typical in North America and Western Europe.

The Six-Year Churn Cycle

Fixed power envelopes create an unexpected consequence for hardware makers: forced silicon turnover. If a data center cannot expand its physical footprint or increase its grid allotment, the only method to increase total intelligence output is swapping out the chips for denser, cooler parts.

“If I have finite numbers of megawatts, I need to replace my chips every few years with better chips,” (play from 1:13:26) Goodwin explains. “And so that's a huge advantage for Fractile as a market entrant because every chip that you see in a data center today will definitely be out of those data centers in six or seven years tops.” (play from 1:13:35)

Incumbent chip architectures like standard HBM and SRAM struggle with memory bandwidth bottlenecks that bleed power. By moving to alternatives like 3D DRAM stacked directly atop logic dies, new market entrants can extract vastly more tokens out of the exact same power drop. The rack space will stay, but the silicon inside it has an expiration date.

What to Do With This

Audit your inference fleet this week by logging tokens generated per megawatt-hour alongside your standard p99 latency metrics. Group your model traffic into real-time interactive queries and asynchronous background tasks. Downclock or re-route background processing to high-efficiency, lower-frequency chip modes to recover up to twenty percent of your power budget without touching user-facing performance.