More Than a Kill Switch

Gates Raises the Alarm

The episode went out on Tuesday 29 September under the title “Bill Gates’s Blunt Warning on A.I.” The show’s own summary is blunter still: Gates thinks AI alarmism “hasn’t gone far enough.”¹ It’s the latest in a run of public interventions since late August, and each one has pushed further than the last.

The first was an essay. On 26 August, Gates posted roughly 6,000 words to his personal site under the title The turbulent AI era is here. The choices we make now are critical.³ It opens on a fork in the road. “AI will either be the greatest equalizer ever invented, or the worst source of injustice,” he writes.⁶ Even if things go well, he expects the transition to be “one of the most turbulent times in human history.” And he says flatly: “There is no plan to ease the entry into the AI era.”⁵

The essay names three dangers. Jobs, because this time the machine is coming for thinking itself. Deliberate harm, from fraud and deepfakes up to cyberattacks and engineered disease. And children, whose resilience and critical thinking he worries will soften in the company of always-agreeable AI. He also puts his own bias on the table, which billionaires rarely do: he still has financial ties to the tech industry, and works with Microsoft and other AI companies through his foundation. “Readers will have to decide for themselves whether this clouds my view,” he writes.⁴

The second intervention came on Sunday 27 September, on NBC’s Meet the Press. The line that travelled was this one: “AI is certainly powerful enough to drive events that, you know, cause a billion deaths.” He followed it with another. “There’s never been a weapon as powerful as the combination of people with ill intent using the latest AI tools.”⁸

Kristen Welker pressed him on the obvious fix. Is a kill switch, a way to turn the thing off, even possible? Gates called the idea “kind of a weird thing.” AI isn’t yet at the point where it can autonomously seize computers and refuse to be shut down, he said. For now the danger is people using it, and people don’t announce themselves. “I would never want to say I’m against a kill switch. But when you’re trying to moderate the bad behavior, you need insight and records of what’s being done. So the kill switch alone would not prevent these tragedies.”⁹

Then came Klein. Here Gates put a date on the danger. “We crossed the cyber threshold and we crossed the bio threshold early this year,” he told him.² On cyber, he pointed to models that are now finding security bugs in code humans had combed over for more than twenty years, and named the run of releases leading up to Anthropic’s Mythos. On biology, he said the ability to design a new pathogen used to belong to nation states. Small groups with the latest tools now have it.²

And the liability argument, the one Klein described as the governing view in Washington? “You can’t rely on the industry to self-regulate here,” Gates said. “I mean, it’s just insane.”²

The Tripwires Go Off

“Threshold” sounds like a figure of speech. It isn’t. The big labs have spent the past three years building written rulebooks that define, in advance, the capability levels at which a model gets more dangerous and has to be handled differently. Think of them as tripwires the companies laid across their own path.

Anthropic calls its version the Responsible Scaling Policy, graded in AI Safety Levels. On 22 May 2025 it switched on its ASL-3 protections for Claude Opus 4, the level meant for models that could give meaningful help to someone trying to build chemical, biological, radiological or nuclear weapons. The company was careful about what it was claiming. It said it had “not yet determined whether Claude Opus 4 has definitively passed the Capabilities Threshold,” but that “clearly ruling out ASL-3 risks is not possible.”¹¹ OpenAI made the same kind of call two months later. In July 2025 it classed ChatGPT agent as high capability in the biological domain under its own Preparedness Framework, calling it a “precautionary approach.” The OpenAI researcher Boaz Barak said “the risk is very real.”¹²

So the tripwires first went off in 2025, and they went off as precautions: the labs couldn’t prove their models were safe, so they behaved as if they weren’t. Gates’ claim this week goes further. In his reading, early 2026 is when the risk stopped being a maybe.

What a safeguard actually is

When a tripwire goes off, what changes? Mostly, a set of extra systems wraps around the model. Anthropic’s centrepiece is what it calls constitutional classifiers: “real-time classifier guards, trained on synthetic data representing harmful and harmless CBRN-related prompts.”¹¹ In plain terms, these are smaller AI models that sit on either side of the big one. One reads what you type before the main model sees it. Another reads what the main model writes before you see it. If either spots a narrow category of weapons help, the exchange is blocked. Around that sit jailbreak detection and, on the security side, more than a hundred controls protecting the model’s weights, including limits on how much data can flow out of the servers where those weights live.¹¹ OpenAI’s package for ChatGPT agent looked similar: refusals, flagging of risky requests for expert review, and monitoring for signs of misuse.¹²

Look at the top row of that diagram and Gates’ objection to the kill switch becomes mechanical rather than rhetorical. A kill switch lives on the model. It can stop it, but it can’t tell you that someone has spent three weeks feeding it innocent-looking pieces of a pathogen design. The classifiers and, above all, the log are what can. That’s the “insight and records” he told NBC about. It’s why his proposal to Klein was not a big red button but what TNW summarised as “a supervisory layer of safeguards and monitoring in all AI models,” strong enough to stop attacks that could “shut down economies or kill millions.”²

Now look at the bottom row. Open-weight models, the ones anyone can download and run on their own hardware, don’t pass through anybody’s classifier. Gates acknowledged this with Klein: anyone can switch off the monitoring on an open-source model. His answer is that open models could stay free and customisable, but should run on platforms the government monitors.² That fits with his January annual letter, where he singled out the risk that “a non-government group will use open source AI tools to design a bioterrorism weapon.”¹⁸ Whether a government can monitor software that runs on a gaming PC in someone’s spare room is a question his proposal doesn’t answer.

There’s a second wrinkle, and Gates raised it himself. The guards are blunt. He told Klein that the filters on Anthropic’s Mythos and Fable models are too restrictive: ask about cancer, he said, and you can find yourself redirected to a smaller model.² This is the heart of the dual-use problem he describes in the essay: “the same AI model that can find a flaw in software so a company can fix it can also help a criminal exploit it.”⁴ Tune the guard too loose and it lets the attacker through. Tune it too tight and it turns away the oncologist. Somebody has to decide where that dial sits. Right now, that somebody is the company.

The Builders Push Back

If Gates is the loudest voice for taking that decision away from the companies, Jensen Huang is the clearest voice for leaving it where it is. Nvidia’s chief executive answered Gates’ jobs warning within days of the essay. “I love the heck out of Bill. But I don’t see what he sees,” he told Fox Business. “Some jobs will be eliminated, but many new jobs will be created.”¹⁴ He pointed to the physical build-out AI is driving. “Don’t forget the chip plants that are being created, packaging, computer plants, and all of the AI factories being created.”¹⁵

In September he went further, on CBS. Asked whether AI could end the world by 2030, Huang said there is “zero percent chance that’s going to be the end of the world.” His prescription was speed, with a condition attached: “We should go as fast as we can irrespective of anybody else,” but “we would never ever, and never should, ship products before they’re ready, deliver products that are unsafe.”¹³ (We weighed that zero against what AI-assisted science actually delivered this year in Zero Percent.)

Huang’s safety model, as reported, is the one Klein put to Gates. Companies already have every incentive and a legal duty not to sell harmful products. If they do, existing liability and cybersecurity law applies. The first job is to enforce the rules already on the books, not to write new AI-specific ones.¹⁶ It’s a respectable position with a long history behind it: most products in most economies are policed this way. And it’s fair to say the obvious out loud. Nvidia sells the chips the frontier runs on, so any slowdown lands on its order book. Gates, for his part, has told us about his own ties. Neither disclosure settles the argument. Both belong beside it.

Gates didn’t always sound like this, and that’s part of why people are listening. On 21 March 2023 he published The Age of AI has begun, an unabashedly bullish essay. A day later, an open letter signed by more than a thousand AI experts and executives called for an urgent pause on systems more powerful than GPT-4. He told Reuters, “I don’t think asking one particular group to pause solves the challenges.”¹⁹ Three years later, he says he would probably support a credible global plan to slow AI down.¹⁰

1. Mar 2023“The Age of AI has begun.” Gates has called AI as revolutionary as the internet and the mobile phone.¹⁹
2. Apr 2023Tells Reuters a pause by one group won’t solve the challenges and would be hard to enforce.¹⁹
3. Jan 2026Annual letter flags open-source bioterror as a risk greater than a natural pandemic.¹⁸
4. 26 Aug 2026The ~6,000-word essay. Three risks, three proposals, one declared conflict of interest.³
5. Aug 2026Huang on Fox Business: “I don’t see what he sees.”¹⁴
6. Sep 2026Gates Foundation commits at least US$1 billion over two years to AI in health, education and farming.^20,22 Huang tells CBS: zero percent.¹³
7. 27 Sep 2026Meet the Press: “a billion deaths”; a kill switch alone is not enough.^8,9
8. 29 Sep 2026The Ezra Klein Show: cyber and bio thresholds crossed; self-regulation “just insane.”²

Figure 3 · From optimist to alarm: Gates on AI, 2023–2026

He also isn’t alone. Anthropic’s Dario Amodei put his own odds of AI going “very, very badly” at 25% in September 2025, and the Nobel laureate Geoffrey Hinton has put the chance of catastrophe at 10 to 20%.^16,21 Hinton’s own case for regulation, and his complaint that the labs are racing to make models smarter rather than nicer, we covered in The Race to Make Them Nicer.

On jobs, both men have some evidence on their side, and it’s worth being exact about what it says. Gates rests his case on work from Stanford’s Digital Economy Lab. Its August 2026 update finds that employment among 22-to-25-year-olds in highly AI-exposed occupations “now stands about 19% below where it would be” had it kept pace with similar workers in less-exposed jobs. The same update also says, in so many words: “We do not see widespread, economy-wide job displacement associated with AI.”¹⁷ The damage is real, and so far it is concentrated at the bottom rung. Huang is looking at the whole ladder. Gates is looking at the rung that’s missing. (For the policy toolkit on that side of the argument, from wage insurance to AI wealth funds, see The Last Payslip.)

The Decision Leaves the Lab

Strip away the headline numbers, the billion deaths and the zero percent, and the two men agree on more than you’d expect. Both say AI will bring enormous gains in medicine and science. Both say unsafe products shouldn’t ship. The fight is over one word in Huang’s sentence: ready. Who decides when a model is ready, and on what evidence?

Gates’ analogy with Klein was drugs, aircraft and cars.² None of those is policed by lawsuits alone. A new drug needs approval before it reaches a pharmacy. A new airliner needs certification before it carries a passenger. Liability still exists in both industries, but it’s the backstop, not the gate. The reason is simple. Liability works after the harm. For most products, that’s tolerable: someone is hurt, a court awards damages, the company fixes the flaw. For the harms Gates describes, an engineered pandemic or an attack that takes down a power grid, there may be nobody left to compensate and nothing left to fix.

What he is actually proposing

The international body is the most ambitious idea. Gates wants something that combines “an inspections regime for nuclear weapons, regulations for international aviation, and agreements that protect the ozone layer.”⁴ Each of those worked because the things being governed were hard to hide: uranium, aircraft, industrial chemicals. Software is easy to hide. The supervisory layer in Figure 1 is his attempt to make it less so.

The tax is the most concrete idea. “If you’re an employer and you hire someone, you pay payroll taxes on their earnings. But if you buy a robot, you can usually write it off right away as a business expense,” he writes. “The tax system nudges you toward replacing people with machines.”⁴ Robots are physical, depreciable and easy to count, so a robot tax is enforceable. A token tax is harder. An AI company’s bill can be taxed. A model running on hardware you own sends no bill to anyone, so a token tax could simply push work towards local models.

The Human Reserved idea is the most personal. Gates compares it to nature reserves, “places where we could put buildings and roads, but we choose not to because the loss would be too great.” It comes from watching paid carers look after his father, who died with Alzheimer’s in 2020. “Something in the care they gave my dad was irreplaceably human,” he writes. He’s honest that it’s unfinished: “The idea of Human Reserved raises a host of questions I don’t have answers to.”⁴

So what comes next isn’t a question about whether AI is dangerous. Almost nobody in this argument, Huang included, says it’s harmless. It’s a question about the machinery. Does a frontier model get checked by the people who built it and sued if they got it wrong? Or does it pass through a layer of classifiers, logs and outside auditors before and after it ships, the way a drug passes through trials? The labs already built the first half of that machinery for themselves in 2025. Gates is asking for the other half, and for someone other than the labs to hold the records.

He gave Klein his answer to the Washington consensus in a single sentence. Huang gave CBS his in a single number. Between the sentence and the number is the actual work, and none of it has been written into law yet.