Anthropic Proposes $1 Billion Harm Threshold for AI Bans

Anthropic, the developer of the Claude artificial intelligence assistant, is seeking to establish a unified federal framework for regulating advanced AI systems in the United States. The company has proposed legislation that defines specific conditions under which the government could prohibit the release or use of an AI model, according to a report by Politico.

High Barriers for Government Intervention

The draft bill stipulates that authorities would be unable to ban an AI model unless it poses an imminent risk capable of causing losses exceeding one billion dollars or resulting in serious injury or death to more than 50 people. This approach aims to set clear safety standards while limiting government restrictions that might hinder technological development.

Scenarios warranting potential intervention include the possibility of AI models being used to manufacture biological weapons, deviating from safety controls established by their developers, or executing cyberattacks. The proposal mandates periodic safety assessments conducted by independent experts and requires companies to publish reports detailing risks associated with advanced models.

Reporting Mechanisms and Legal Procedures

Under the suggested framework, firms must implement safety plans and protect employees who report security issues. Serious incidents must also be reported. If relevant government agencies identify an imminent danger meeting the specified criteria, they are required to notify the U.S. Department of Commerce within 72 hours of completing their assessment.

This notification enables the department to request a federal judge in Washington to halt the model’s operation. The initiative emerges amid growing concerns regarding the ability of AI agents to scan websites, discover security vulnerabilities, or exploit them.

Federal Preemption and State Conflicts

Anthropic attempts to balance oversight of AI risks with establishing clear boundaries on government authority to block new models. However, the proposal raises questions about whether the defined harm thresholds are excessively high and if authorities should intervene before catastrophic losses occur, particularly since some dangers may remain severe without reaching the cited figures.

The bill also seeks to unify federal rules and reduce certain requirements found in state laws regarding risk disclosure, independent evaluations, and incident reporting. This consolidation could spark disputes over the limits of federal power versus the role of states in protecting the public.

Anthropic confirmed that the proposal reflects its own AI safety framework. The legislation remains unapproved, and it is not yet clear whether the administration of President Donald Trump will adopt it.