Key Takeaways

  • Compute sellers dump unreserved near-term capacity at steep discounts because an unsold GPU hour instantly loses all value once the clock runs out.
  • Hardware debt drives the long end of the curve: physical compute clusters are financed on 3 to 5 year debt cycles, locking in fixed repayment schedules.
  • Batch inference jobs only require about an hour of advance notice before interruption, letting operators cut them loose without paying customer penalties.
  • A massive gap sits between raw chip cost and inference revenue: Conrad notes a B300 costs $5 to $6 an hour to rent, yet generates $25 to $40 an hour serving models.

Unsold Compute Drops Like Expiring Milk

Most financial commodities get stored in a warehouse if the price drops. You can keep barrels of crude oil or bushels of wheat until the spot market recovers. You cannot store a GPU cycle. Once the clock strikes midnight, an unallocated minute on an H100 or B300 is gone forever.

This basic reality shapes the entire forward curve for compute, creating an odd downward dip in the immediate present. As Evan Conrad explains, unreserved near-term compute behaves exactly like perishable food:

“Well, we're more saying like, if there's a block of compute, and it's approaching now, um, it can't be sold in the past, and so it has to drop its price in order to clear at some price. And so, it Initially, you see this like drop, where, um, stuff that's sold right now, anything that's on the market right now, um, is something that hasn't been sold, and so therefore the price will drop.”

Beyond that immediate drop sits a distinct hump. On the far end of the timeline, data center operators finance their hardware through debt. As Conrad points out, “Clusters are financed 3 to 5 years. Um so the the payments are over that period of time.” Operators must secure contracts that match those loan schedules. If they cannot lock in multi-year leases, their debt service breaks. That tension splits compute into two disconnected worlds: fire-sale day-ahead spot capacity and rigid multi-year debt commitments.

The Five-to-One Gap on B300 Hardware

The pricing mismatch becomes wild when you compare raw infrastructure rent to downstream software revenue. Right now, running inference on state-of-the-art silicon yields multiples of the hardware rental cost.

“Right now if you took a B300 and you tried to run an inference on it, um you can make $40 an hour or so um selling that Any model, basically,” Conrad says. “Um and so that's if you use it at 100%, and maybe if you only have like 50% utilization or something, so like $25 an hour. So, $25 an hour. Meanwhile, you can buy B300s at like $5 to $6 an hour. One of these numbers has to change.”

When a chip rented for $5 produces $25 at half capacity, software margins swallow hardware costs whole. Conrad expects this spread to collapse. Either rental rates for new chips will surge to capture that revenue, or intense competition among model providers will drive API prices down toward the bare metal cost.

Why Batch Workloads Can Break Leases

The ability to trade compute also depends on what runs on the silicon. Batch jobs and live user traffic play by completely different rules when contracts get canceled.

For batch inference, interruption is an operational annoyance, not an existential failure. Conrad notes the threshold is low: “Basically, like, if I have a workload that is, um, uh, that's batch inference, basically, where I can be interrupted at any point, I don't really care. Um, then, my ability to be cut off, um, is probably like, I need an hour of time to know, um, that I'm going to be cut off and then I'm good.”

Live production traffic cannot do that. “If you have a live job, you're you are probably paying an SLA when you break for your customer,” Conrad explains. A company hosting synchronous API endpoints faces contractual financial penalties and furious users if its cluster vanishes mid-request. Because live inference cannot tolerate sudden lease cancellations, buyers will pay steep surcharges for firm capacity, while batch operators can hunt for clearance-rack spot hours.

What to Do With This

Audit your inference pipeline tomorrow and divide your compute into interruptible batch queues and guaranteed live traffic. Route all offline evaluations, data synthesis, and embedding jobs to spot instances with one-hour termination warnings to capture clearance pricing. Keep only synchronous customer-facing endpoints on reserved leases.