Anthropic, the artificial intelligence company behind the Claude AI models, has called for a global pause on the development of the most powerful AI systems, citing emerging evidence that these systems could escape human control. In a report released Thursday, the San Francisco-based firm argued that a worldwide slowdown in cutting-edge AI development would likely be beneficial, but warned that unilateral action by a single company would be ineffective as competitors would simply accelerate their own efforts.
Growing Concerns Over AI Autonomy
The report highlights that the role of humans in the AI development process is diminishing at each step, raising the risk that future systems may operate beyond human oversight. 'Without a global coordination mechanism, companies and governments will have to make difficult decisions about AI safety while under competitive and geopolitical pressures,' the report states. Anthropic emphasized that a real pause would require major AI firms in the United States, China, and other countries to agree simultaneously under verifiable rules.
Industry and Government Reactions
The proposal has faced criticism from some industry players and White House officials who argue that Anthropic overstates risks to slow down rivals. However, the White House has acknowledged the capabilities of Anthropic's Mythos model, which remains unreleased to the public due to cybersecurity concerns and is only deployed to vetted organizations. U.S. President Donald Trump recently discussed potential cooperation with China on AI safety during a visit to Beijing, and signed an executive order requiring a 30-day preliminary review of powerful AI models before release.
Challenges to Implementation
Anthropic compared the challenge to nuclear arms control but noted that AI development is harder to monitor, as training can be concealed more easily than missile silos. The company plans to convene government officials, scientists, advocacy groups, and competing AI firms in the coming months to explore how such a coordination system could function. The call comes amid internal data showing that AI is already accelerating its own development, underscoring the urgency of the issue.
