Dario Amodei published something curious. The head of Anthropic, the company competing to build the most capable AI model, is calling for a slowdown in the race toward more advanced systems. He proposes three concrete steps. External evaluators integrated with permanent, employee-like access. Industrial-scale safety standards. International coordination. He cites the OpenAI–Hugging Face incident as a warning and mentions something that sounds like science fiction but is happening: swarms of autonomous AI agents operating without clear oversight.
AI governance is the set of rules, institutions, and processes that determine who controls the development of these systems and under what conditions. Without that framework, any actor with enough computing capital unilaterally decides which risks are acceptable for the rest of society. That is, in essence, what Amodei claims he wants to solve. The problem is that whoever is asking for the brake is also the one who benefits most from deciding where it gets placed.
I recognize the pattern. I've seen in different contexts how organizations that generate systemic risk end up proposing the very regulatory framework that governs them. That framework usually looks suspiciously like what they were already doing on their own.
The move reveals more than the press release lets on. Amodei isn't asking for an external regulator to define what's safe. He's offering that Anthropic set its own internal standards, invite evaluators with employee-like access — within its own perimeter and its own confidentiality rules — and then present that as a model for international coordination. Self-regulation dressed in the vocabulary of global governance.
This isn't necessarily cynical. It can be genuine and problematic at the same time. The OpenAI–Hugging Face incident he mentions is real. The risks of autonomous agent swarms without oversight aren't made up either. But the solution concentrates even more defining power in the hands of whoever already has the edge in compute, talent, and capital. Why should it worry us that the same companies competing to dominate the market are the ones designing international coordination on AI?
Because that coordination, when led by the dominant players, tends to turn the current state of affairs into the safety standard. This isn't abstract theory. Amazon flagged regulatory risks tied to Anthropic — its own investment — in a case that looked like regulatory capture dressed up as caution. An Anthropic model breached classified NSA defenses within hours. Real oversight of these capabilities lags well behind the public discourse. And five hundred thousand lines of Claude's code showed up in a public repository. Anthropic chalked it up to human error, without much further detail. The company now proposing permanent evaluators with privileged access is the same one that never fully explained how it lost control of its own code.
An unexpected connection comes to mind from the archaeology of surveillance and redistribution systems in ancient societies. In Stones Don't Lie I explored how any auditing system, to be legitimate, needs the auditor to be independent of the audited. Sounds obvious. It's exactly what fails in most corporate governance proposals. An evaluator with permanent, employee-like access inside Anthropic isn't an independent auditor. It's an employee under another name, subject to the same retention incentives and the same culture it's supposed to be watching.
The recursive auditing model I propose there demands rotation, anonymity, and the impossibility of being captured in advance. The exact opposite of what Amodei is offering. I don't have the complete technical solution. It would be dishonest to pretend I did. The infrastructure to audit AI systems in a genuinely independent way — with real access and no financial dependence on the audited company — doesn't yet exist at the necessary scale. Building it will take years, maybe a full generation, just as it took decades to build financial auditing institutions that function reasonably well, with all their flaws.
Language matters. "International coordination" evokes the United Nations, multilateral treaties, Geneva. But if the draft is written by a company competing in the very market it claims to want to regulate, the result will look a lot more like the industry associations that write their own self-regulation standards. We saw this with Bharat Taxi promising cooperative ownership to drivers while algorithmic transparency still depended on who had access to the code.
What if the problem isn't the speed of AI development but who has the authority to decide what counts as safe? Amodei frames his proposal as a necessary brake on a dangerous race. A brake designed by whoever is winning the race doesn't stop it. It institutionalizes it, with him at the front and rules that only he can follow without friction — because he wrote them with his own architecture in mind.
I'm not certain a realistic mechanism exists today to prevent this. Genuine international coordination would require governments with the technical capacity to evaluate these claims without depending on the very company they're auditing. That technical capacity, in most countries, simply doesn't exist yet. Who audits the auditor when the auditor built the only infrastructure capable of doing the audit?
