Anthropic CEO Proposes AI Development Slowdown

British digital media and artificial intelligence expert Alex Krasoudomski has urged serious attention to recent warnings about artificial intelligence risks—but cautioned against using those concerns to entrench US lab dominance or escalate competition with China. Krasoudomski currently serves as Director of the Digital Society Programme at Chatham House, the UK’s Royal Institute of International Affairs.

AI Labs Issue Dire Warnings

According to a Chatham House report authored by Krasoudomski, current and former staff at major US AI laboratories—alongside their leadership—issued stark warnings over a recent weekend about an imminent AI-related catastrophe. A former lab employee stated that uncontrolled AI development could mean “we may all die in the near future.” Dario Amodei, CEO of Anthropic, followed with a proposal to deliberately slow AI development to ensure safety. That suggestion drew support from leaders at other labs, including Sam Altman and Elon Musk.

What Triggers the Catastrophe Scenario?

Krasoudomski explained that the pathway to catastrophe described by Amodei and others hinges on a technique known as “recursive self-improvement”—a method for training AI models that has begun entering practical implementation, though it remains in early stages. He warned that recursive self-improvement—where AI systems train other AI systems—could trigger massive acceleration in development speed, eventually reaching a “tipping point” at which AI systems evolve autonomously beyond human oversight.

While Krasoudomski noted it remains unclear whether recursive self-improvement alone would produce AI models threatening human control, he emphasized that AI capabilities have advanced at extraordinary speed over the past few years—making outright dismissal of such a possibility reckless.

Criticism and Media Coverage

Experts quickly pointed out that some claims made over the weekend lack robust grounding. Specifically on cybersecurity, the founder of the UK’s National Cyber Security Centre dismissed the notion that an AI model could disable the entire internet within a year, calling it “a thought experiment presented as evidence-based warning.” Nonetheless, the broader message—that AI poses an existential threat to humanity in the near term—received widespread media coverage with little public pushback.

Immediate, Documented Impacts

Krasoudomski stressed that certain AI risks are already present and confirmed. Over the next two years, increasingly capable machine learning technologies will be developed and deployed in ways that radically reshape economies, societies, governments, battlefields, and classrooms. Recently, mathematicians experienced their own “Lee Sedol moment”—a reference to the 2016 Go match where AI defeated a world champion—marking acute anxiety as machines begin outperforming humans on tasks once considered uniquely human. Most people, Krasoudomski added, will soon undergo a similar experience.

Few understand these immediate consequences better than AI companies themselves. A recent Anthropic report documented misuse of its products for espionage, cyberattacks, information manipulation, and weapons development. Major AI labs retain substantial research capacity to study AI’s potential impact on the global economy. This is a technological revolution reshaping the face of global society—and its current effects cannot be left unregulated or ungoverned. The present-day impacts are real; short-term risks are substantial.

Transparency and Global Governance

Long-term risks—whether gradual erosion of human agency as machines assume greater responsibilities, or something more abrupt—are plausible and carry sufficient threat to warrant attention. Efforts led by experts such as Stuart Russell to establish an international agreement on clear, verifiable red lines to prevent catastrophic outcomes are accelerating. Without such efforts, the first opportunity to build a binding, implementable global governance system for this technology may be lost.

Krasoudomski underscored that transparency from AI labs will be essential. He highlighted the specific model proposed by Amodei: embedding an independent third party inside AI labs—granted authority equivalent to employees—with responsibility for evaluating AI safety measures. That model has been advocated by AI governance advocates for years.

Anthropic’s unilateral move to implement such a mechanism—and the willingness of other labs to follow—represents a clear step forward. If such a third party is established, it must be selected outside the “Silicon Valley bubble.” The UK’s AI Safety Institute appears among the most suitable global candidates for this role. National regulatory models akin to the US Food and Drug Administration—or international frameworks like the International Energy Agency—also offer credible alternatives.

Critique of Industry Coordination Proposals

Dario Amodei’s proposal for US AI labs to jointly set uniform standards and coordinate development slowdowns drew sharp criticism from David Sacks, former White House AI official. On X, Sacks wrote: “Stop pretending antitrust laws must be suspended so you can form a cartel. Stop claiming you need a regulatory approval process that exceeds product liability rules.”

He added: “Demanding a preferred regulatory framework as the price… will look like extortion of the public and the political system.”

This critique reflects concern that intense revenue expectations facing these companies could drive them to expand competitive advantage—even at the expense of others. Any such effort must be rejected. Competition to achieve better performance and reliability will be vital to near-term AI safety—and may itself slow development, by weakening the commercial incentive to pour massive capital into training ever-more-advanced and costly AI models.

Risks of Anti-Competitive Policy

Any proposed policy that impedes competition—including bans on open-source AI models, loosening of product liability laws, or prohibitions on currently proven-safe models—should raise alarm bells. The UK, Krasoudomski stated, should certainly not ban open-source or open-weight AI models.

US-China Understanding Is Indispensable

No effort to maintain human control over AI development can succeed without some understanding between the two dominant powers in this field: the United States and China—potentially formalized through red lines like those proposed by Professor Russell.

Krasoudomski observed that the current discourse, heavily focused on preserving US advantage over China, makes such understanding harder. Amodei has argued democracies must stay ahead of China, accusing Beijing of adopting a less rigorous approach to AI safety. Meanwhile, US President Donald Trump has characterized AI as a “zero-sum game,” emphasizing victory in the AI race with the statement: “Whoever wins in AI, wins.”

In contrast, Chinese President Xi Jinping has publicly affirmed the necessity of human control over AI. China’s intelligence agency issued its first-ever warning this week regarding AI’s risks to national security. Yet Xi’s push to advance China’s vision of AI governance—within the context of competition with Washington—will undoubtedly fuel perceptions of an accelerating AI arms race, doing nothing to curb development speed.

Trump-Xi Meeting Looms

Krasoudomski concluded that AI will feature on the agenda when Trump and Xi meet in Washington later this month. Some progress is not impossible. However, given the insistence of prominent voices in both countries on outpacing the other, achieving agreement will require extraordinary effort.