Key Takeaways

  • Software development is splitting into two distinct roles: product engineers who define requirements in issue trackers, and factory engineers who build the systems that write the code.
  • Warp dropped its internal cost per automated pull request from $80 to $15–$20 by adding model routing, benchmarking test runs, and continuous agent evaluation.
  • Product engineers will rarely edit code directly in an IDE, spending their time instead in Slack, Linear, Jira, and GitHub issues.
  • Factory engineering turns software production from a subjective guessing game into an observable, measurable pipeline with clear error rates.

The Split Between Spec Writers and System Architects

Most software teams still treat coding agents like faster auto-complete. A developer sits inside an editor, prompts an agent, waits, fixes the syntax errors, and pushes a commit.

Warp founder Zach Lloyd argues this setup is temporary. As autonomous coding factories handle entire tasks from intake to pull request, the traditional developer role splits into two separate disciplines.

“The way to think about using the factory is that there's actually sort of two personas,” (play from 5:32) Lloyd said. “There's someone who is building the product, they're doing product engineering, they're shipping features through a factory.” (play from 5:40)

These product engineers do not spend their days reading stack traces. They write user requirements, clarify edge cases, and run acceptance checks. Their primary workstations are Linear, Jira, Slack, and GitHub. They trigger tasks via APIs or Model Context Protocol connections, treating the coding factory as an external team that works at machine speed.

On the other side stand the factory engineers. They do not write customer-facing features. Instead, they act as platform architects. Their job is building and maintaining the automated pipeline that turns high-level specs into tested pull requests. They design the agent prompts, manage context windows, build the evaluation checks, and write the deterministic tests that catch broken runs.

How Warp Cut the Cost of a Pull Request to Fifteen Dollars

When a team first connects an autonomous agent to a repository, the immediate temptation is to send every step to the most capable, expensive frontier model. The bill adds up fast.

Warp initially spent roughly $80 in model compute for each pull request generated by their system. They cut that cost to between $15 and $20 by treating code generation as an industrial optimization problem rather than a raw compute spend.

They accomplished this by setting up three operational pillars: model routing, benchmark evaluation environments, and real-time monitoring. Routing sends routine jobs like file linting and doc generation to small, cheap models, reserving top-tier models for ambiguous architectural planning. Automated test environments run pull requests against unit tests, checking whether generated code actually builds before a human ever looks at it.

“The really big picture thing that I think is so cool and powerful here is like you're getting out of the realm of guessing, and you're getting into what we call factory engineering,” (play from 18:02) Lloyd explained. “You are taking a real engineering mindset. You're like, how do I... We have this amazing new primitive of coding agents. How do I know as a team, as a company, that we're actually deploying them in the right way?” (play from 18:12)

Factory engineering requires treating your codebase like an assembly line. When an agent fails, you do not blame the model. You inspect the failure, update the automated test suite, improve repository documentation, or adjust the routing logic so the mistake cannot happen twice.

What to Do With This

Assign one engineer on your team to spend Friday building an automated evaluation test for a single repetitive bug fix. Have them write a script that sends a past GitHub issue to an agent, checks whether the agent's pull request passes your continuous integration suite, and logs the token cost of the attempt. Measure the pass rate across five runs so your team has an exact baseline cost per fix.