Key Takeaways

  • Warp cut its internal cost per pull request from $80 to between $15 and $20 by routing models dynamically and benchmarking agent performance.
  • Ad-hoc terminal scripts create fragile developer setups that fail across machines and resist automated testing.
  • Defining a factory in a declarative YAML file provides version control, rollbacks, and lets agents submit pull requests against their own workflows.
  • Agentic factory infrastructure becomes practical once an engineering team reaches 5 to 10 developers.
  • Zach Lloyd organizes this architecture around Zach Lloyd's 8-Layer Software Factory Infrastructure Stack.

The Zach Lloyd's 8-Layer Software Factory Infrastructure Stack

Most engineering teams run agents as isolated local experiments. Zach Lloyd saw that this creates fragile workflows: “When I talk to companies or developers, most of the world is trying to build some sort of internal software factory. What I think it means is like a genetic system in the cloud where you're defining what are the roles of various agents across building software for your team or for your product and there's a certain amount of autonomy in that system.” (play from 2:20)

To make those loops reliable, teams must define their automation as code. “You should actually think of a software factory as a very specific thing,” (play from 3:43) Lloyd said. “You should define your factory in code. You should make sure that the factory has all of the workflows, skills, MCPs. Everything is like it's like a concrete thing that you should be able to run at some point in time and measure how it's doing.” (play from 3:47)

At Warp, that foundation lives in version control: “At the bottom of the stack, I think it's really foundational and important that you define your factory declaratively as code. The analogy is like Terraform here.” (play from 19:40)

Lloyd structures that foundation into eight layers:

  • Layer 1: Declarative Code Definition: Define your factory declaratively as code (similar to Terraform) in a version-controlled YAML specification that outlines repositories, integrations, and workflow stages, allowing agents to submit pull requests against the factory definition itself.
  • Layer 2: Centralized Observability & Artifact Store: Gather all agent conversations, prompt iterations, computer use screenshots, and artifacts across tools (Slack, Linear, GitHub) into a single system of record to diagnose failures and replay runs.
  • Layer 3: Integration & Access-Controlled MCP Layer: Plug in Model Context Protocol (MCP) servers and tools with granular permission controls partitioned across organizational boundaries (e.g., engineering, marketing, sales).
  • Layer 4: Pluggable Compute Sandbox Layer: Isolate execution inside modular execution sandboxes (e.g., Name Space, Daytona, Modal) utilizing standard Linux environments for backend tasks or virtualized macOS instances for simulator testing and computer use.
  • Layer 5: Flexible Inference & Harness Layer: Support multiple model providers to burn down committed enterprise spend alongside ACP-compatible harnesses (e.g., Claude Code, Codex, Pi).
  • Layer 6: Continuous Improvement & Eval Layer: Deploy LLM-as-a-judge scorers to monitor online agent quality, catch failure patterns (like test bloat), and run self-optimization feedback loops.
  • Layer 7: Orchestration Engine: Direct work along stages (triage, spec, implementation), load appropriate skills, select models, and trigger workflow automations.
  • Layer 8: Multi-Modal Knowledge Work Access Points: Expose headless, API- and MCP-driven interfaces embedded directly into team workflows, including Slack, Linear, Jira, GitHub comments, and CLI terminals.

When This Works (and When It Doesn't)

This framework applies to engineering organizations with at least 5 to 10 developers seeking to standardize and scale agentic development. At that size, uncoordinated agent usage leads to fragmented code styles and runaway API bills. Warp lowered its cost per pull request from $80 down to $15 to $20 by routing models and evaluating agent runs inside this shared setup.

It is overkill for 1-to-3 person teams who move faster with individual local client subscriptions. Early-stage founders lose speed if they spend their time maintaining virtualized sandboxes and orchestration pipelines instead of shipping code.

What to Do With This

If you lead an engineering team of eight developers who are expensing separate AI coding tools, set up Layer 1 this week.

Create a repository named software-factory and commit a factory.yaml file. Declare your active repositories, list authorized MCP servers (such as GitHub, Linear, and your test database), and outline a three-step path: triage issue, generate reproduction test, open draft pull request. Put that file behind branch protection so every workflow change requires peer review. Next week, prompt an agent to add a linter check to that specification and review its pull request.