Key Takeaways

  • Jason Lemkin claims open-source model inference peaked because frontier closed providers cut their API prices.
  • Modal CEO Erik Bernhardsson rejects the claim as fixed-pie thinking, projecting both open-source and closed inference will surge 5x to 20x over the next 12 months.
  • Bernhardsson admitted Modal made a strategic mistake by waiting on the sidelines instead of allocating compute to language model inference earlier.
  • Open-source creators are adopting commercial licensing models, giving enterprise buyers custom control over latency, throughput, and weights.
  • Bernhardsson projects that specialized inference across focused models will build a 100 billion dollar company.

The Disagreement

Harry Stebbings brought a common Silicon Valley talking point to the table: Jason Lemkin's argument that open-source model inference has already hit its ceiling. The reasoning sounds clean on paper. When frontier labs slash API prices to pennies per million tokens, the economic argument for running your own weights collapses. Why deal with hosting, cluster management, and GPU supply when you can call a proprietary endpoint for cheap?

Erik Bernhardsson sees that argument as flawed. “I think that's a little bit of a fixed pie size mentality,” (play from 38:48) Bernhardsson said. “With all due respect I do think both spaces will grow 5x 10x 20x over the next 12 months and beyond.” (play from 38:53)

The conflict comes down to how you view buyer demand. Lemkin treats model inference as a zero-sum budget where cheaper closed models cannibalize open weights. Bernhardsson sees an explosion of untapped demand that frontier APIs cannot satisfy on their own.

Bernhardsson learned this lesson the hard way at Modal. “I think we should have allocated more resources towards LM inference earlier,” (play from 37:50) he admitted. “We're a little bit on the sidelines for a while. We didn't realize how big open source inference would be.” (play from 37:54)

Who's Right (and When They're Wrong)

Lemkin is right if your workload treats models as fungible commodities. If your product only needs basic text generation or simple classification, self-hosting makes no financial sense. You take on GPU availability risk, engineering overhead, and hardware management just to match the output quality of a cheap API call. If token price is your only metric, closed endpoints will win every time.

Bernhardsson is right everywhere else. Production systems at scale do not buy tokens based solely on price per million. They buy based on predictability, custom latency, throughput, and data privacy.

“There's always going to be people who want to fine-tune, want to have more control over their models, and sort of tweak the parameters, like go low latency, high throughput,” (play from 27:44) Bernhardsson explained. Closed APIs give you a black box. You cannot adjust batch sizes, tune speculative decoding, or run quantization to hit sub-50-millisecond response times.

Open-source model creators are also changing how they monetize. “Let's also remember the open-source models, they're all moving into licensing models,” (play from 27:16) Bernhardsson noted. “They're getting better at capturing the value of those models.” (play from 27:21)

That commercial shift matters. When model builders capture revenue through enterprise licenses rather than relying on pure charity, they guarantee continuous development. Bernhardsson sees this dynamic unlocking massive enterprise value: “I think there is an opportunity to create a hundred billion dollar company with a lot of specialized models in inference.” (play from 0:00)

The market is not picking one architecture. Startups prototype on closed frontier models because setup takes two minutes. Once traffic scales and margins tighten, engineering teams move high-volume workflows to fine-tuned open models where they can shave milliseconds and control their unit economics.

What to Do With This

Audit your model architecture this week by splitting your LLM calls into two buckets: exploratory queries and repetitive, high-frequency workflows. Leave low-volume and ambiguous reasoning tasks on closed commercial APIs. For any endpoint generating more than 100,000 predictable requests per day, pull a sample dataset and benchmark a fine-tuned open-weight model on dedicated infrastructure to calculate your true latency and margin gains.