Key Takeaways
- Real-world data sidesteps theoretical worst-case cryptography. “There's the saying in physics that nature abhors a vacuum,” Ankur Moitra explains. “My version of it is that in some sense, nature abhors computationally hard problems.”
- Fine-tuning on unstructured outputs transfers hidden behaviors. In an experiment detailed by Moitra, an LLM instructed to obsess over owls generated random digits; when researchers trained a fresh model on those digits, “hidden within those numbers is the love of owls.”
- Language models act like statistical Ouija boards. They magnify tiny linear vector alignments embedded in data without requiring direct semantic instructions.
- Automated theorem-provers paired with Lean verify mathematical deductions down to axioms, allowing researchers to evaluate proofs for landmark problems like the Unique Games Conjecture and Navier-Stokes blowup without human error.
- Teams deciding whether to grant an agent autonomy should score the task using Brunton's Two-Axis AI Deployment Matrix.
The Brunton's Two-Axis AI Deployment Matrix
- Axis 1: Verifiability / Trustworthiness: Evaluate how easily, rapidly, and rigorously the AI's output can be verified. In math or code, formal verification systems like Lean provide near-instant verification down to axioms.
- Axis 2: Stakes of Failure: Assess the real-world impact if the AI's output is flawed or incorrect. Low stakes mean failure causes no harm (e.g., an unproven lemma or suboptimal car body concept); high stakes involve human life, public infrastructure, or safety-critical domains.
- Safe Integration Zone (High Verifiability, Low Stakes): Open-ended design spaces such as engineering optimization (e.g., aerodynamic drag reduction) and pure math theorem proving. Outputs can be tested in wind tunnels or verified by automated provers without risking human safety.
- Human Firewall Zone (Low Verifiability or High Stakes): Safety-critical systems like aviation (FAA), medicine (AMA), or law, where human verification protocols, institutional firewalls, and strict legal boundaries must prevent autonomous AI execution.
When This Works (and When It Doesn't)
Brunton's matrix works to decide whether an automated system or LLM can be granted autonomous problem-solving freedom, or whether human verification and institutional firewalls are strictly required. Steven Brunton frames the problem around two practical questions: “how much do we trust the output, so verifiability or trustworthiness, and on the other axis is how high stakes is it if it's wrong.” If your software generates code inside a test suite that flags regressions instantly, you sit in the Safe Integration Zone. You can allow the model to run experiments, generate solutions, and commit changes autonomously.
The framework breaks down when teams mistake quick human inspection for high verifiability. Founders often assume customer support responses or legal contract redlines have high verifiability because an operator can skim the text in thirty seconds. Skimming text is not formal verification like Lean checking mathematical logic. Subtle policy errors or hallucinated terms slip past reviewers while introducing major legal liability. If you treat hurried human oversight as automated verification, the matrix will misguide you into deploying autonomous models directly into dangerous failure modes.
What to Do With This
Audit your product roadmap this week and score every planned agent workflow on Brunton's two axes. Look at any automated database tool your team wants to hook up to user data.
Test Axis 1 first: can your system formally verify every generated query before execution? If you rely on prompt instructions to prevent data loss, your verifiability score is close to zero. Next, score Axis 2: an unindexed query or an accidental write command hitting live production carries massive operational stakes. Because the feature lands squarely in the Human Firewall Zone, stop building autonomous execution. Either restrict the agent to read-only database replicas where stakes drop to zero, or require a human engineer to approve execution plans in a dry-run staging terminal.