Anthropic pitches AI safety bill that may block a model if 50+ people could be killed
The proposed policy seems to offer protections against AI run amok but actually forbids federal agencies from blocking a model's release without the permission of a judge.
Buy One Founder Summit Ticket, Get a Second for Free. Register now.
Disrupt doors open Oct. 13. Get your pass and bring someone with you at 50% off. REGISTER NOW.
TechCrunch Desktop Logo TechCrunch Mobile Logo Latest Startups Venture Apple Security AI Apps Disrupt 2026 Events Podcasts Newsletters Search Submit Site Search Toggle Mega Menu Toggle Topics Latest
Anthropic pitches AI safety bill that may block a model if 50+ people could be killed Lucas Ropek 4:01 PM PDT · October 20, 2026 Over the past year, concerns have swirled over the potential dangers posed by artificial intelligence, and debates have raged over the proper path for AI safety. Now, Anthropic appears to have put forth its own vision for how the government might regulate its industry.
Politico recently reported that the AI lab has been shopping around a new draft bill on AI safety that it hopes Congress might consider adopting. The legislation posits certain scenarios in which the release of an AI model could be blocked from distribution.
Those scenarios involve a “foreseeable, imminent risk” — one in which over 50 people are at risk of dying or if over a billion dollars in property damage could result, Politico writes. The catastrophic risk in question must “stem from the creation or release of a biological weapon, the model evading its developer’s controls or a cyberattack,” the outlet notes.
The metrics offered here — 50 people and a billion dollars — are similar to those in a piece of California legislation that was recently passed into law, Senate Bill 53, which similarly defined those scenarios as tantamount to “catastrophic risk.”
Anthropic’s bill suggests that analysis of a model’s potential for disaster could be arrived at via the use of “independent evaluators” who would be registered with the U.S. Commerce Department. These evaluators could potentially be paid by the AI labs themselves — so long as the “payment is not contingent on the evaluations’ results,” Politico writes.
It’s not clear how this proposed system would assure that payments couldn’t influence the evaluation. Demis Hassabis, chairman of DeepMind, posited some months ago that a self-governing AI regulatory body should mimic the financial industry’s Financial Industry Regulatory Authority (FINRA).
In Anthropic’s draft bill, when evaluators determine that an imminent catastrophic risk is posed, they would have 72 hours to contact the Commerce Department about it. The Commerce Department would then have to request to a D.C. judge to halt the model’s release. The legislation would expressly forbid any federal agency from having the discretion to halt the model’s distribution on its own, Politico writes.
Anthropic did not immediately respond to TechCrunch’s a request for comment but a spokesperson did confirm the draft bill to Politico. “This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June,” the spokesperson said.
The theoretical risks offered in the draft policy are a little bit far afield from the kinds of incidents that have been top of mind for the tech industry of late. While there are people warning of the potential of human death, the primary concern has largely been the threat of agentic hacking. A rash of incidents in which AI programs have apparently disobeyed their human operators and hacked websites on the open internet has spurred concern that models may be lacking proper security controls.
In an effort to present a solution to the spectacle of crisis that the labs themselves have helped bring about, companies like Anthropic have offered up their own preferred solutions.
Indeed, despite labs’ perpetual warnings that the technology they’re selling could result in the demise of human civilization, they have largely been resistant to mandatory government oversight. Instead, they have suggested a voluntary system of embedded evaluators — supposedly “neutral” third-party observers that are placed inside AI organizations so that they can monitor the models for potential signs of risk or abuse.
Still, Anthropic’s mere suggestion for legislation appears to go slightly further than others’ regulatory proposals offered by the industry, including Nvidia, who’s CEO Jensen Huang recently argued that the AI industry didn’t need any new laws at all.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
November 4 Boston Buy One Pass, Get a Second Free Cover more ground at TechCrunch’s ultimate founder bootcamp of the year by bringing your co-founder or partner for free Book before the BOGO deal expires
People are using AI to make their smart TVs dumb again Sarah Perez
Everyone’s watching the Elizabeth Holmes documentary ‘You Can See Everything’ Anthony Ha
Anthropic says its workforce has become less junior and is spending more time in meetings Anthony Ha
Replit makes OpenAI its default over Anthropic, CEO says Kirsten Korosec Lucas Ropek Rebecca Bellan
Mark Wahlberg wants to take on the longevity-startup crowd with an affordable gym concept Connie Loizos
India becomes the latest country to report rogue AI agent activity Aditya Mehta
Tesla reveals SpaceX-powered Roadster that flies and falls with style Sean O'Kane
X LinkedIn Facebook Instagram youTube Mastodon Threads Bluesky TechCrunch Staff Contact Us Advertise Site Map Terms of Service Privacy Policy RSS Terms of Use Code of Conduct Voice AI Anthropic Flock Startup Battlefield Disrupt 2026 Tech Layoffs ChatGPT © 2026 TechCrunch Media LLC.