Key Takeaways

  • Software development is the only sector displaying genuine labor substitution from large language models today.
  • Code provides tight gradient descent with objective ground truth: scripts execute or crash, giving models an instant feedback loop for recursive self-improvement.
  • Enterprise tasks like legal drafting and slide creation lack ground truth, meaning autonomous model loops produce exploding errors instead of polished output.
  • Automated coding tools are driving a surge in software vulnerabilities, handing windfalls to reinsurers and cybersecurity vendors rather than generating enterprise-wide savings.

Code Has Ground Truth That Office Work Lacks

Every tech executive projecting massive enterprise revenue from AI points to software engineers. They watch a developer generate a pull request in seconds and assume corporate finance, legal, and operations departments will soon follow suit.

Paul Kedrosky of SK Ventures calls this a fatal misread of how software works. Software engineering has an unusual mechanical property: tight gradient descent backed by definitive ground truth. You write a function, run it against test suites, and the interpreter gives you an immediate binary answer. It works, or it fails.

Because of that clear pass-fail signal, automated systems can run unattended. Kedrosky explains: “software is really unusual in another way, which is that it has a what's called a tight gradient descent, which means that the feedback loop, as you let it work in its own undirected way at whether it's RSI, recursive self-improvement, or it's something more mundane like just improving some old financial code, you can let it loose and do that because there's ground truth, right?” (play from 49:23)

When you step outside a compiler, ground truth vanishes. A pitch deck, an M&A memo, or a business strategy does not have a compiler to reject bad logic.

The Early Adopter Trap in Venture Capital

Projecting broad office adoption from developer behavior repeats a classic venture capital blunder.

“One of the tenets of venture capital is that your early customers look nothing like your later customers,” (play from 48:20) Kedrosky notes. “So if you project a business based on the early adopters of your product, you're making a terrible mistake.” (play from 48:25)

Software engineers are unusual early adopters because machines can evaluate their core deliverable. When Anthropic and other foundation model providers attempt to justify massive capital investments, their forecasts demand massive enterprise seat penetration. Kedrosky points out the revenue math: “The only way to get there is to assume deep penetration of the enterprise with Anthropic and mass substitution, because there's only two ways you can generate the revenue. You're either substituting for other things we're already doing, or we're complementing it and making them so much more productive that the whole pie grows.” (play from 47:51)

Yet software engineering remains the single discipline showing that substitution effect. Applying that playbook to knowledge workers who deal in ambiguity hits an immediate wall.

Compounding Errors and the Hidden Security Tax

When you remove ground truth from recursive loops, systems degrade quickly. “So if you let models loop recursively on things that don't have ground truth, you get exploding errors,” (play from 49:57) Kedrosky warns. “You don't get better product, and this is a deep problem as you transition from coders to normal users.” (play from 50:01)

Instead of autonomous agents drafting strategy, white-collar staff end up babysitting hallucinations. Without a compiler to catch small hallucinations, each iteration compounds the initial flaw.

Even in software development, the productivity boom carries a hidden bill. Fast automated generation does not mean secure generation. Automated code generation has triggered an explosion in critical software vulnerabilities and cybersecurity incidents across enterprise repositories. Instead of netting pure cost savings, companies find their budget diverted to security software vendors and cyber reinsurers to clean up technical debt.

What to Do With This

Audit your product roadmap this week. If you are building autonomous agents for unstructured tasks like marketing copy or strategy, stop relying on multi-step prompt loops without human validation gates. Confine autonomous iteration strictly to workflows where a script, compiler, or deterministic test suite can verify ground truth after every run.