AI Governance Just Became a Contract Question

Anthropic builds some of the most capable AI models in the world. On September 12, 2026, its chief executive published an essay asking the industry, his own company included, to build them more slowly.

The piece is titled "We Must Pace the Frontier." Dario Amodei argues that frontier labs should deliberately slow the rate at which they increase model capabilities. He also revisits a pause proposal he once waved off. It "made little sense back then," he writes, because models could not yet act coherently as autonomous agents. They can now. What follows is the clearest look anyone has had at AI governance written by the people who build the models. It reads like a company volunteering to be watched.

"We must slow the pace at which we improve the capabilities of AI models." Dario Amodei, September 12, 2026.

Amodei's plan has 3 steps, and 1 of them is already under contract

Step one is embedded evaluators. Frontier labs would give independent assessors, METR among them, permanent employee-level access. Desks, badges, company laptops. The standing right to examine safety practices and publish what they find, with redactions limited to security-sensitive material.

Step two is a common safety standard, adopted across labs in democracies through regulation or voluntary industry agreement. It comes with capability checkpoints. A model's safety properties have to be certified before it can be released. Step three is democracies negotiating shared limits with governments that are not democracies.

Anthropic has contractually bound itself to step one. Sam Altman said employee-like access for outside evaluators "is a great idea, and we will do the same." Elon Musk's public answer to the essay was "Dario is right." Satya Nadella welcomed deliberate pacing and embedded evaluators in concept.

An endorsement can be delivered in a sentence. A contract has to be signed by somebody who can be held to it.

Here is where each company stands today.

A published document is the cheap part

Factories bought electric motors decades before productivity moved. Paul David's research on the dynamo traced the delay to the rebuild. The motor was available long before the floor was redesigned around it, and the people working that floor had to learn a different job before any of it paid.

Most of us have been handed a policy at work that changed nothing. Most of us have also been handed one that changed the following Monday. The difference showed up in whether anyone outside the team was allowed to check.

That is what makes today's announcement from Microsoft worth reading closely. Microsoft published its Code of Conduct on September 14, 2026. It governs the company's own MAI models, 7 of them, spanning reasoning, coding, image, voice, and transcription. Microsoft opened the document for public consultation the same day. Nadella's framing was blunt: "If it's not under human control, it's not worth pursuing." He added that the work "cannot be controlled by a handful of entities," a generous line from the head of one of the handful.

The rule-making is moving to where the models are

Every institution touching this technology is working from rules written for a slower world. That now includes the companies building the models. A release cycle measured in months outruns any review process built around an annual calendar, internal ones included. Opening a governing document to public comment on the day it publishes is an attempt to close that distance in the open, where anyone can read the draft and argue with it.

Human control is the phrase here that a reader outside the industry can hold onto. It means somebody who can tell when a model is wrong. Judgment, taste, trust, accountability: the capacities I call Above the Algorithm in my forthcoming book, The Great Relearning. An evaluator with a badge is one version of that. A company that keeps enough experienced people to read the evaluator's findings and act on them is the other. The second one cannot be purchased.

California took a different route to the same problem this month, paying outside verifiers through a new auditor registry under SB 813 and AB 1405. Congress has its own version of the access question already on the table, in the AI Kill Switch Act's push to define who can stop a model and how fast. Different levers, same instinct: put somebody independent in a position to check.

These read as corporate housekeeping decisions. They set the terms for how fast the work millions of families depend on gets rearranged. They also set how much warning anybody gets first. A safety standard adopted at a frontier lab this month shows up later at somebody's desk, as whether a claims processor or a junior developer has time to relearn the work.

I spent 20 years at Cisco shaping a $1.1B innovation portfolio. In every review that mattered, the commitments that held were the ones somebody had put in writing and attached a name to. The rest were intentions. Intentions hold up right until a quarter goes badly.

So here is the question I am carrying into my own conversations this month. Of everything your company has said publicly about how it builds or buys AI, how much has a name on it, a date, and one person outside the team allowed to check? Write that list. If you do not set the policy where you work, the question still belongs to you, and asking it in your next team meeting costs you nothing.

Four of the largest AI companies in the world spent this week arguing about their own speed limits in public. The rest of us get to read the drafts while they are still drafts. That is a better week than most. If you read it differently, I'm easy to find.

Here is what makes Alex a credible voice on this topic: he advises the California State University system on AI governance, where the question these labs are debating this week, who outside the team is allowed to check the work, has to be answered in practice rather than in an essay.

Put the same question to your own AI commitments before somebody else does. Book a conversation →