Key Takeaways

  • Neocloud GPU hosting is commoditizing rapidly because competent hosting providers offer almost zero technical differentiation once clusters are online.
  • Model creators protect software margins by co-designing the runtime stack. DeepSeek achieved higher throughput and faster interactivity on identical GPUs by custom-building their inference server.
  • Silicon holds defensibility due to physics and multi-year design lead times. Fractile targets inference bottlenecks by 3D stacking DRAM directly atop logic dies instead of relying on standard High Bandwidth Memory (HBM) or SRAM.
  • Fractile chose to stay purely at the chip layer and avoid competing with customers, bypassing intermediaries like Broadcom and refusing to monetize internal models.

Neoclouds Race to Zero on GPU Hosting

Billions of venture capital poured into GPU clouds over the past two years. Startups leased clusters of H100s, wrapped them in basic Kubernetes dashboards, and called themselves AI infrastructure companies. That margin window is closing fast.

Walter Goodwin, founder and CEO of AI chipmaker Fractile, points out that raw compute reselling has no structural defense: “What is commoditizing and I don't think anybody has ever argued it won't be is kind of you know the whole space of of cloud compute. You know one neocloud that is competent is much like another neocloud that is competent.” (play from 8:03)

If your service sells vanilla GPU hours, you are selling electricity and rack space. Your customer will migrate the hour another provider drops pricing by five percent. When hardware is identical, margins bleed until only the providers with the cheapest energy and lowest cost of capital survive.

Software Co-Design and the Hardware Moat

Defensible value is pooling at two distinct edges of the AI hierarchy: the physical silicon and the frontier model runtime.

At the software layer, running an open-weight model out of the box leaves performance on the table. Goodwin highlights DeepSeek to illustrate how software engineering creates a moat: “Deepseek had done so well in building an inference stack to run their models on GPUs that they were getting, you know, much higher throughputs, much better interactivity, better speeds than than everybody else who had built a Deepseek inference server.” (play from 7:17)

DeepSeek did not just release model weights. They engineered the kernels, memory allocation, and serving infrastructure specifically for the underlying hardware. Third-party hosts running generic vLLM setups could not match their speed or cost profile.

At the physical layer, chip design demands multi-year capital bets constrained by hard physics. Goodwin explains: “I think one of the reasons why, you know, I am very long on chips in terms of where that value occurs or being one big place where that value occurs is actually because of the nature of the challenge of placing the right chip bet.” (play from 52:05) Inference workloads are bottlenecked by memory bandwidth, not raw math compute. Moving bytes from off-chip memory consumes time and power. To solve this, Fractile bypasses traditional packaging setups and 3D stacks DRAM directly on logic dies.

Because the hardware takes years to build, winning architectures cannot be copied overnight by a software release. To keep their incentives aligned with their buyers, Fractile maintains clear boundaries: “we never intend to compete with our customers. we will never monetize internal models because that's how we've chosen to build this company. We've chosen to compete in the chip layer.” (play from 3:11)

What to Do With This

Audit your compute bill this week. If you are paying a premium to a neocloud simply to rent raw GPUs, benchmark your workloads across three competing providers to negotiate your rate down immediately. If you serve models in production, stop using stock serving libraries; assign an engineer to profile your memory bandwidth and write custom kernels for your specific batch sizes to double your throughput on existing hardware.