Key Takeaways
- Dario Amodei and Demis Hassabis want a self-regulatory organization (SRO) modeled after FINRA, which David Sacks warns will turn into a bureaucratic "DMV for AI."
- Sacks proposes an independent ratings board modeled on the MPAA, which formed without government oversight to replace the Hays Code.
- Chamath Palihapitiya argues that real safety requires opening intermediate reasoning tokens, because closed API models hide their internal alignment checks.
- Open-source architectures let developers inspect model reasoning directly, exposing alignment failures before deployment.
The Disagreement
Frontier AI labs want a formal body to police model safety before weights hit the public. Anthropic CEO Dario Amodei and Google DeepMind CEO Demis Hassabis support creating a FINRA-style self-regulatory organization. In finance, FINRA writes rules, licenses broker-dealers, and enforces compliance under government authority. Proponents argue this creates clear guardrails without waiting for gridlocked legislatures to draft technical rules.
David Sacks sees an anti-competitive trap. Giving leading labs the power to approve models before release creates an artificial bottleneck that protects incumbents from smaller, faster competitors.
“I call it a DMV for AI because I think what's going to happen is all these models are going to get lined up in a queue waiting to get their test done and then released and it's going to slow us down horribly,” Sacks said. “And what I'm against is these companies meeting in secret whether it's part of some sort of like FINRA or otherwise to set standards that are not transparent that are uncontestable. And you know what's going to come out of that? Regulatory capture and anti-competitive behavior.”
Instead of a quasi-governmental regulator, Sacks points to Hollywood. In 1968, movie studios abandoned the mandatory Hays Code and formed the Motion Picture Association of America (MPAA) ratings system. The MPAA does not report to the state. It issues voluntary letter grades (G, PG, R) and lets consumers and theater owners decide what to show.
Chamath Palihapitiya points out that true safety audits are impossible as long as commercial model makers conceal their internal reasoning steps behind closed endpoints.
“There is one thing that you must do if you believe that this is true to further steel man this thing. You cannot obfiscate the thinking tokens of a model,” Palihapitiya said. “And the thing with the open source models is you can actually see it thinking. So you can see its misalignment in real time. And so instead we have to basically agree to their interpretation of tokens that we can't see that only they can see that they can interpret how they want.”
Who's Right (and When They're Wrong)
Sacks and Palihapitiya identify the main threat: giving closed frontier labs the authority to decide who can ship software. When regulators rely on proprietary companies to define safety tests, the resulting standards favor the companies with the biggest compliance budgets.
The MPAA model works well for content warning and commercial classification, but it breaks down on genuine dual-use dangers. If a model lowers the barrier to biological threats, voluntary labeling after release will not fix the damage. However, hiding the reasoning chain behind an API does not solve safety either. It merely centralizes trust in four or five corporate boards.
Palihapitiya is right that visibility beats pre-clearance. Inspecting chain-of-thought tokens reveals where a model rationalizes harmful actions. Without access to those hidden tokens, any external audit (whether from FINRA or an independent board) is pure theater.
What to Do With This
Audit your product's reliance on closed API models that conceal intermediate reasoning. If your system depends on reliable logic, test an open-weights alternative where you can log and inspect full chain-of-thought traces. Do not base your architectural roadmap on the assumption that closed-source API access will remain cheap, open, or unmonitored.