Key Takeaways

  • GPU lead times currently sit at three to four months, forcing infrastructure teams to forecast demand through exponential or straight-line aggregate customer extrapolation.
  • Buying hardware at industry conferences feels like chaotic street trading. Bernhardsson described Nvidia's March GTC gathering in San Jose as “an open air drug market, but GPUs instead of drugs.” (play from 16:40)
  • Modal bypasses off-the-shelf orchestrators like Kubernetes, choosing instead to write its own container runtime, file system, and scheduler to pool servers across 25 separate neo-clouds.
  • Hardware pricing follows gravity. While supply constraints may inflate compute costs for another one to five years, prices will inevitably collapse over the long term.

Hardware Prices Obey Gravity

When you build an AI company during a compute crunch, high prices feel permanent. Founders lock themselves into rigid multi-year capacity contracts at peak valuations because they fear getting locked out of compute entirely. That fear creates bad unit economics.

Bernhardsson takes a long view on the hardware squeeze. “Hardware prices like it's like gravity. Hardware prices always go down in the very long run,” (play from 20:58) he notes. Current market conditions distort how builders think about infrastructure. High prices reflect short-term scarcity, not permanent structural costs. “Right now we're in a very you know there there's a supply shortage and you know that means that prices have gone up. They may continue to go up for another year, two years, 5 years I don't know but eventually they will come down.” (play from 21:01)

If you treat temporary hardware premiums as permanent realities, you will build business models that collapse the moment supply catches up with demand.

Aggregating 25 Neo-Clouds Without Kubernetes

Most teams try to manage infrastructure by deploying Kubernetes clusters across major providers. Modal went in the opposite direction. They built a custom software layer to stitch together fragmented compute from 25 different neo-cloud providers into a single unified fabric.

“We built our own container files. We built our own file system. We built our own container runtime. We built our own scheduler. We built our own basically how we managed the capacity,” (play from 17:35) Bernhardsson explains. “And that that enables us to to basically treat like all of the world's compute as one big pool of capacity.” (play from 17:45)

This architecture solves two acute problems. First, it absorbs spiky customer usage by spreading execution across dozens of independent suppliers. Second, it protects Modal from the three-to-four-month order backlogs common in the current GPU supply chain. Instead of committing to massive upfront hardware purchases from a single vendor, Modal uses aggregate customer trend lines to acquire capacity incrementally across a fractured vendor landscape.

What to Do With This

Audit your compute commitments this week. If you are signing contracts with terms longer than twelve months, demand floating price adjustments tied to market hardware rates rather than fixed fees. If a vendor refuses, split your workload across on-demand neo-clouds instead of over-committing to one provider at peak pricing.