Key Takeaways

  • Less than 30% of enterprise production tokens (roughly 28%) now run through proprietary frontier models, with open-weight models absorbing the rest.
  • Engineering teams evaluated by Code & Co. build dynamic routing architectures that send 75% to 80% of routine traffic to smaller models, reserving frontier systems for the top 20% to 30% of complex queries.
  • Jim Milbery argues that retrofitting open-weight models after token costs blow out gross margins is fireable malpractice for a CTO, not operational agility.
  • Kirby Montgomery points out that widespread customer abandonment of low-utility chatbots has temporarily masked unit economic risks across portfolio companies.

The Disagreement

The tension sits between two views on operational tolerance in software engineering.

Kirby Montgomery describes a reactive workflow common across private equity portfolios: engineering teams deploy features on frontier models like Claude or OpenAI to test market appetite. Then, when monthly vendor invoices surge, they re-engineer the stack for open-weight alternatives to repair unit economics.

Jim Milbery has zero patience for that progression. “The fact that they now say, 'Hey, we got to move this to open weight' means, yeah, you should have been fired as the CTO, the product guy,” Milbery says. “Like if you knew that a zillion of your customers needed to use this and you knew going into it that you needed to be able to scale it without expensive tokens.”

Milbery treats API-first generative AI features as an operational failure if the team never stress-tested token costs at scale. For him, a CTO who leaves unit economics to post-launch discovery has failed the job.

Dan Bender reframes the challenge around architecture rather than model loyalty. Bender cites data showing that under 30% (roughly 28%) of production tokens now run through frontier models, with open-weight models absorbing the rest. Starting on a commercial API during customer discovery is normal. The real problem is shipping production software without an intelligent traffic router.

Who's Right (and When They're Wrong)

Milbery is right when assessing core product workflows with predictable usage. If a software company automates data extraction or document processing across thousands of daily transactions, piping raw queries straight to a proprietary frontier model destroys gross margins on day one. A management team that waits for the cloud bill before addressing cost architecture deserves Milbery's assessment.

Bender and Montgomery are right during early product validation, but only because so many generative features fail. As Montgomery observes, “over the last year a lot of worthless AI features have been built. A lot of chat bots are out there that nobody uses.” Spending engineering cycles to fine-tune and host an open-weight model for a feature that customers ultimately abandon burns cash faster than commercial API tokens ever could.

The operational line comes down to routing maturity. Bender notes that top teams reserve frontier models for the top 20% or 30% of complex queries, while routing 75% or 80% to smaller, cheaper open-weight models. When diligence reveals a company hardcoding proprietary API calls into core workflows without dynamic routing, the company exposes itself to unhedged margin dilution.

Why It Matters

Software multiples historically assumed 80% gross margins. Blind reliance on frontier model APIs pushes margins down toward professional services economics as feature usage expands. Deal teams evaluating add-on acquisitions and platform tech debt must treat dynamic query routing as a baseline requirement for software margin defensibility, rather than an optimization left for exit preparation.