Key Takeaways

  • North Korea used AI to accelerate a cyber attack on Binance that netted more than $5 billion, pushing AI-enabled theft past 5% of North Korea's entire GDP.
  • Open-weight models distilled from American frontier models distribute automated cyber warfare tools globally with no kill switch.
  • Patrick McKenzie estimates that cleaning up vulnerable legacy code will require at least $500 billion in automated cyber remediation.
  • Frontier AI labs face a conflict of interest: their releases power automated offensive probing, while their commercial enterprise divisions sell the defensive patches to fix it.

The Disagreement

When a state actor drains billions using automated exploits, who pays the bill?

Zvi Mowshowitz argues that frontier labs bear direct responsibility for foreign attacks. If an American lab trains a model that can find zero-day vulnerabilities, foreign competitors will copy and distill those weights. Once an adversary strips the guardrails, they point that capability at global infrastructure. Mowshowitz puts the blame squarely on the developers: “Will the labs be blamed for this? I mean, somewhat sure, but they should be blamed for this, right? Like there would not be these Chinese models that are capable of doing this without the American models.” (play from 1:16:07)

In Mowshowitz's view, the labs unleashed an asymmetric weapon. Attackers no longer need human teams writing exploits manually. As Mowshowitz points out, “And now that you can probe and exploit at scale in parallel... any number of people can do it and those capabilities are going up, and they can concentrate their firepower on whatever they decide the weak spot might be at any time.” (play from 1:15:24)

Patrick McKenzie shifts the focus from moral blame to structural economics. Distillation cannot be undone. As McKenzie notes: “And once the open source models and the open weights achieve a capability, that is the worst capabilities will ever be going forward. Unless you like successfully do a global-wide surveillance regime over all like basic computing, which is hopefully impossible.” (play from 1:18:13)

Because the capability is permanent, assigning blame to the lab does not solve the underlying exposure. McKenzie points out the bizarre business model this creates: legacy codebases across banks, utilities, and governments have accumulated decades of security debt. Fixing that legacy exposure requires at least $500 billion in automated software remediation. The labs that built the offensive capability are now positioned to capture that $500 billion market by selling the automated defense.

Who's Right (and When They're Wrong)

Mowshowitz is right on causation, but wrong on legal enforceability. American labs did provide the training blueprints and synthetic tokens that allowed foreign teams to distill offensive capabilities. But tort law rarely holds an upstream software manufacturer liable when a foreign military uses an altered version of their code to commit grand theft. If regulators try to impose joint liability on labs for open-weight derivatives, labs will simply move their research offshore or lobby for total bans on public model releases.

McKenzie is right about the practical outcome. You cannot litigate an exploit out of an open-weight model running on an offline server in Pyongyang or Beijing. The baseline floor for offensive capability has permanently shifted upward. The offense-defense asymmetry means that any system exposed to the public internet will face continuous, parallel probing from cheap local models.

The real failure mode is assuming defense moves as fast as offense. North Korea showed that a $5 billion theft can happen overnight, while updating enterprise legacy software takes years. The labs will clean up financially by selling defensive audit agents, but the victims will foot the bill long before the fixes ship.

What to Do With This

Audit your public endpoints for automated probe resistance by Friday. Run an open-source security agent against your staging environment to see how many legacy API endpoints it flags within three hours, then patch the top three unauthenticated paths before an automated scanner finds them.