New York City Proposes an AI Kill Switch and Invites Five CEOs to the Hearing

Any AI system marketed, sold or deployed in New York City would need outside validation and a human-operated kill switch under a bill Council Speaker Julie Menin introduced on September 25. A business or validator caught operating without that validation, or with a falsified one, would owe $25,000 per instance.

The rest of the package goes further. A second Menin bill would pay whistleblowers a portion of the fines levied against AI companies that break applicable laws, an approach the Council calls first in the nation. A third, from Council Member Virginia Maloney, would let people harmed by malicious AI use sue the company behind it. And on October 5, the full Council, all 51 members sitting as the Committee of the Whole, has asked Sam Altman, Dario Amodei, Sundar Pichai, Elon Musk and Mark Zuckerberg to attend its hearing.

I read that invitation list twice. Five of the most powerful people in technology, asked to sit before a city council. A Council source says negotiations with several of the companies are active, and the Council holds subpoena authority if it needs it.

What the three bills would do

The validation bill carries the most technical weight. Before an AI system can be marketed, offered for sale or deployed in New York City, an outside validator would review its data quality, bias, privacy, security and outputs, and check it against NYC Cyber Command requirements. The system would also need a kill switch, meaning a human-override capability that lets a person shut it down.

Maloney's private right of action sets a specific test. A person harmed by malicious AI use could sue when the harm was foreseeable, reasonable safeguards were missing, and a third party exploited that failure. The bill names jailbreaking explicitly, the practice of tricking a model into ignoring its own safety rules.

The whistleblower bill pulls a different lever. The announcement didn't detail specific reward percentages, but the structure is clear: the person inside a company who notices a violation gets a share of what the company pays for it. Around these three sit smaller measures, including 24-hour incident reporting to Cyber Command for city agencies and contractors, emergency response planning for AI disruptions, protections for city employees and contractors who speak up, disclosure requirements, a ban on false safety claims, and privacy and security rules for chatbots.

Rules written for a slower world

Every institution that governs technology works on its own clock. A bill gets drafted, heard, amended, voted on and put into practice, and each step takes the time careful deliberation needs. The systems under discussion ship updates in weeks. I call this pattern rules written for a slower world, and companies live inside it too: plenty of corporate AI policies approved last spring describe tools the team has already replaced, the same pattern I found running through a recent survey of companies that bypass their own AI rules under deadline pressure.

What stands out in the New York package is its choice of tools. A kill switch and a whistleblower reward both hold up after the technology changes. A human override works the same way whichever model is running next quarter. A reward for insiders puts the people closest to a system on watch, whatever version they happen to be testing. The Council picked rules built to travel with the technology.

Every AI company has someone who sees the problem first: an engineer who notices a model leaking private chat data, or a tester who finds a jailbreak before launch and then sits with that knowledge, weighing what speaking up would cost. Under the Menin bills, that person would have a financial reason to come forward, and city employees and contractors would have explicit protection for doing so. Speaker Menin put the Council's purpose plainly.

We now have an even greater responsibility to ensure that we have the appropriate safeguards in place to protect New Yorkers.

The hard part is the definitions

Here is where the real work sits, and where the Council can draw on help. "Validation" and "kill switch" are clear words in a press release. Turning them into requirements a validator can test and a company can meet takes detail. I advise the California State University system on AI governance, and this is the stage where a policy either becomes a practice people follow or stays a document. What counts as a biased output? Who qualifies as a validator? How fast does a kill switch have to work, and who is trained to reach for it, the same question NIST's work on agent identity is trying to answer for anyone building the systems in the first place? When one system serves many customers at once, whose hand is on the override?

Those questions have answers, and many of the people who can supply them already work in New York: security engineers, auditors, university researchers and the companies themselves. A hearing with five CEOs invited is one way to start that work. Written technical comment from the people who build and test these systems is another. The Council has put a concrete draft on the table early, which gives everyone with real expertise something specific to improve.

A kill switch is only as good as the person trained to use it.

What leaders should take from October 5

If your company sells or deploys AI in New York City, read these bills as a preview. Could your team name the person who can shut your system down today, and say how long it would take? Could you hand an outside validator your documentation on data, bias testing, security and outputs within a week?

For many teams, the answer to both is "not yet," and the reason is people. The skills to validate and override AI systems sit with a small number of employees, and many organizations have put off training anyone else. I call that the learning debt: deferred retraining that compounds until a rule or a crisis calls it due. New York's proposal puts a price on that debt: $25,000 per instance.

And if you never build AI at all, this package still belongs to you. The chatbot your kid talks to after homework would carry privacy and security requirements under it. When you follow the October 5 hearing, ask your own council member one question: who, exactly, can switch it off?

I expect other cities to read these bills closely, because they give every governing body a working draft to study. The kill switch is a small idea with a large question inside it. Somebody has to know how to press it. Who in your organization would that be? If you are working through that question, I'm easy to find.

Here is what makes Alex a credible voice on this topic: Alex advises the California State University system on AI governance, the same work of turning a rule on paper into a step people can actually follow that New York City's kill-switch bill now has to do.

Talk with Alex about what it takes to make an AI kill switch actually work →