Key Takeaways

  • Warp dropped its autonomous pull request generation cost from $80 to $15-$20 by pruning redundant subagents and assigning cheaper models to simple tasks.
  • Running top-tier models like Claude Opus on issue triage wasted budget without improving code quality.
  • Model routing and architecture tuning lifted Warp's pull request throughput by 40% over one month.
  • Metered API access is 5 to 50 times more expensive than provider subscription tiers, where a $200 plan can yield $8,000 in API compute.
  • Zach Lloyd argues that agent companies cannot survive as token resellers; defensibility comes from workflow optimization and agent productivity.

Why Warp Spent $80 on Single Pull Requests

When Warp launched its automated software factory on internal repositories, the initial bills were unsustainable. Generating a single pull request cost about $80. The system produced functional code, but the unit economics failed basic business logic.

The high cost stemmed from two mistakes: model misallocation and agent sprawl. Warp assigned frontier models to low-complexity tasks. Zach Lloyd explains what happened: “when we first started using the factory approach, it was costing us about $80 per PR because we were using like Opus where we shouldn't have. Our triage agent was using like a really high-powered model. You know, we were doing a bunch of redundant work with our the shape of agents. Like we were doing too many sub-agents relative to just having fewer agents.” (play from 24:03)

Spawning multiple subagents triggered compounding token loops. Each subagent spent tokens reading files, summarizing state, and reporting back to parent agents. When a triage agent uses Claude Opus simply to route an incoming issue, budget disappears before anyone writes code.

How Model Routing Lifted Engineering Throughput

Warp fixed its economics by treating agent setup as an engineering pipeline. The team built continuous evaluations, defined repositories and tools with Model Context Protocols (MCPs), and routed tasks based on difficulty.

Triage, file indexing, and basic syntax tasks went to fast, cheap models. Frontier models were strictly reserved for complex implementation steps. Warp also replaced sprawling subagent trees with fewer, direct agents operating on clear boundaries.

The financial and operational returns followed immediately. Lloyd reported: “we were able to get the cost per PR down to like 20 bucks or 15 bucks, something like that. Well, we can measurably improve the throughput of our PRs. Over the last month, it's gone up by 40%.” (play from 24:31)

Cutting spend by 75% actually accelerated development. Reducing agent layers eliminated coordination latency and stopped models from repeating each other's work.

Why Token Resellers Lose to Subsidies

Warp's cost curve shows why building an AI business around token markups is dangerous. Many agent builders buy raw API access, add a margin, and sell it to teams. Swyx warned that this model breaks against vendor subsidies: “I think a lot of people go straight to API of pay by API and that is between 5 to 50 times more expensive than just getting a whole bunch of subs.” (play from 31:10) He pointed out the math: “for like a $200 plan you get 8,000 of API.” (play from 32:12)

Model labs subsidize seat subscriptions to acquire users. If an engineer gets thousands in compute for a flat monthly fee, metered token billing cannot compete.

Lloyd recognized this trap early: “The cool thing about warp factories is like we're actually not trying to pitch like a token reselling product. We're trying to pitch basically a coding agent optimization platform and a coding agent productivity platform.” (play from 32:53) The value lies in orchestrating workflows, managing context protocols, and routing work effectively.

What to Do With This

Audit your agent token spend by workflow step this week. Replace frontier models on classification, issue triage, and context discovery with lighter models. If your system spawns nested subagents for simple tasks, collapse them into a single agent running with a strict prompt and direct repository access.