Anthropic calls for global pause in AI development, warns of 'self-improving' systems

Anthropic has issued an urgent call for a coordinated global pause in the development of advanced AI systems, warning that the technology may be nearing a stage where models could begin to improve themselves without direct human intervention.

According to Anthropic, the most advanced frontier models are approaching capabilities that could allow them to refine their own performance, either by generating improved versions of themselves or by significantly automating the research process behind AI development.

It argues that while today’s systems still rely heavily on human engineers, the trajectory of progress points to a future where AI could play a direct role in accelerating its own improvement cycle.

This possibility, often referred to in research circles as “recursive improvement,” is seen by some experts as a key threshold in AI development, one that could dramatically increase both capability and risk.

In its appeal, Anthropic is urging governments and leading AI developers to consider a temporary global pause on training the most advanced models until stronger safety frameworks are in place.

The company warns that unilateral action by individual firms is insufficient, since competitive pressure among major AI developers could drive continued rapid deployment even without adequate safeguards.

Instead, it is calling for coordinated international agreement on limits, testing requirements, and evaluation standards for frontier AI systems.

Anthropic’s call underscores a growing divide within the artificial intelligence sector.

While some companies and researchers advocate rapid deployment to maximise economic and scientific benefits, others argue that development is outpacing the ability to properly understand and control risks.

Supporters of stronger regulation warn that advanced AI systems could be misused or behave unpredictably if deployed without sufficient oversight. Critics of a pause, however, contend that slowing development could allow less regulated actors to gain an advantage or reduce innovation momentum.

What “self‑improving AI” means

The concern raised by Anthropic centres on the idea that AI systems may eventually assist in designing improved versions of themselves. This could include writing or optimising code for future models, automating parts of AI research and training, and improving the efficiency of model architectures.

While current systems do not yet operate independently at this level, incremental progress in automation and model capability has raised questions about where the boundary lies.

The United States (US), Europe, and Asia have already begun introducing frameworks for AI safety, transparency, and risk assessment; Kenya is still lacking. However, there is no unified global standard governing frontier AI development.