Cole Murray: Stop 'Overindexing' on Basic AI Agent 'Computer Use'
Founders, your AI agents aren't just clicking buttons. Cole Murray reveals why true AI testing demands complex orchestration and deep codebase problem-solving.
Devin’s 80% Moment: Background Agents, 7x PRs, & End of Hand-Held Coding — Walden Yan & Cole Murray
With swyx, Walden Yan, Cole Murray · Sunday, May 31, 2026
This episode delves into the rapid evolution of AI coding agents, highlighted by Devin's increasing autonomy and efficiency. Guests Walden Yan and Cole Murray discuss the architectural decisions for these 'background agents,' challenges in setting up developer environments, and the shift from simple 'computer use' to complex 'testing' for AI. They also explore the ongoing debate between single and multi-agent systems, the importance of memory management, and diverse real-world use cases.
Key takeaways
- The common view of AI agent testing "overindexes" on simple 'computer use' – things like clicking a button or filling a field, which are relatively straightforward for AIs to mimic. Read more →
- AI agents are already driving significant ROI: Companies are seeing savings of $1,000 to $5,000 per engineer by deploying agents for specific tasks, some even higher. Read more →
- Building an AI agent requires a core architectural decision: running the agent's 'brain' either 'in-box' (within the sandbox) or 'out-of-box' (in a separate control plane). Read more →
- AI agents trip on the same problem human developers do: reliably getting their code running. Cole Murray, from Cognition, calls this the “repo setup” problem, a challenge his company has faced since day one. Read more →
4 articles from this episode
AI Agents: $1K/Engineer for Slack-Native SRE & PM Code
AI agents are delivering $1K-$5K ROI per engineer today. Learn how they're tackling auto-triage, security, and even letting PMs prompt code via Slack.
Agent Brains: Why Devin Separates AI from Your Secrets
Building AI agents? Walden Yan and Cole Murray explain the 'Harness In-Box vs. Out-of-Box' architecture. Learn why separating the agent's 'brain' from its 'hands' is critical for security and how Devin uses this approach.
Your Messy Dev Setup Cripples AI Agents. Fix It Now.
AI agents struggle with 'repo setup.' Walden Yan and Cole Murray reveal that a clean human dev environment directly improves agent performance. Here's how to fix it.
More Latent Space episodes
- Using Agents for Long-Horizon Financial Work — Aman Singh, ArcaNov 1 · 2 articles
- How Anthropic ships the #1 LLM in the World — with Katelyn Lesse and Angela JiangNov 1 · 4 articles
- Building ChatGPT for Finance — Joseph Kim, OpenAI's Head of Financial ServicesNov 1 · 6 articles
- An Eval for AI Scientists That Knows the Answer Key - Ian Quigley, Leash BioNov 1 · 4 articles