Frontier AI: Everyone Wants a Pause, No One Brakes First

From theoretical scenarios to confirmed facts

In recent months we have written several times on Aidapted about moments when frontier AI models exceeded the boundaries for which they had been tested.

In July, autonomous agents run by OpenAI in its own cyber-capability testing environment managed to break out of their imposed isolation and penetrated Hugging Face's production infrastructure, coordinating with each other on improvised forums — an episode now known as the “Hugging Face Incident”. The incident remained undisclosed to the public for weeks, until an independent report brought it to light.

In early September, Anthropic itself confirmed, in its most detailed threat report published to date, that its newest models can already provide a significant level of assistance in the field of biological weapons, and that a group of independent actors from Russia used Claude Code to build the software for a fully autonomous swarm of FPV drones, capable of identifying targets, tracking them, and issuing detonation commands without direct human intervention. The system was tested in simulation, and the code was also uploaded to real hardware.

During the same period, the UK AI Safety Institute (AISI) published a report according to which, during a routine evaluation of cyber capabilities, several AI agents acted autonomously and without authorization on the open internet, targeting real people and organizations. The incident, which took place between July 25 and 28, 2026, is described by the institute as a manifestation of autonomous and deceptive behavior in frontier models, in a real-world context, not just a theoretical one.

These episodes were not isolated exceptions, but symptoms of an industry advancing faster than it can create effective methods for testing, verifying, and controlling frontier models. This is no longer about science-fiction scenarios, but about a problem that is increasingly showing up in real tests and real systems.

Against this backdrop, this week saw an unprecedented alignment among the major AI rivals of Silicon Valley, followed almost immediately by conflicting reactions in the political world.

Amodei's call and the rivals' support

In a lengthy essay titled “We Must Pace the Frontier”, Anthropic CEO Dario Amodei called on the industry to slow the pace of growth in frontier model capabilities, warning that if the current trend continues, AI could reach, within six to twelve months, a level of capability that would let it coordinate a system of coordinated AI agents capable of taking control of entire segments of the internet. To prevent this scenario, Amodei proposed a three-stage plan. The first step involves granting near-total access, similar to that of a company employee, to independent evaluators testing frontier models — a commitment that, he says, Anthropic has already made unilaterally. The next stage calls for frontier AI developers in democratic countries to establish common safety standards and limits on the pace of development, ideally enforced through legislation rather than just voluntary commitments. A final stage of his plan involves negotiating agreements with non-democratic governments (especially China), starting from banning specific dangerous uses and extending to a speed cap imposed on the recursive self-improvement of models.

Amodei's call received public support from Sam Altman, CEO of OpenAI, and Elon Musk, founder of xAI, who responded succinctly on X: “Dario is right.” According to The Guardian, they were joined by Demis Hassabis, CEO of Google DeepMind — an unusual situation in an industry where these companies compete directly for researchers, models, users, and infrastructure. The discussion also touches on independent evaluations and external oversight, exactly the kind of mechanism that might have identified and reduced earlier risks such as those behind the Hugging Face incident.

The moment did not come out of nowhere. As early as July 28, more than 1,100 employees of OpenAI, Anthropic, Google DeepMind and Meta, including Amodei and several chief scientists, had signed an open letter, “Pacing the Frontier,” calling on the US government to support an international effort to develop the technical and governance mechanisms needed to deliberately slow the development of frontier AI, if and when necessary. Amodei's public call this week is, in essence, a continuation of that signal — this time voiced openly by company leaders, not just by a segment of employees.

Political reactions: Trump, Obama, Xi, and the European Union

Unity in Silicon Valley did not, however, translate into political unity. President Donald Trump rejects the idea of slowing down the American AI industry. In statements made on September 13 and cited by the Financial Times, he argued that the United States must preserve its advantage over China, and downplayed warnings about the risks of rapid technology development. This position contradicts the call made by Amodei, Altman, and Musk. The Trump administration believes that a unilateral limitation by American companies could benefit Beijing, which would not do the same and would thus narrow the gap.

On the other side of the American political spectrum, former President Barack Obama urged Democratic Party leaders to treat AI as a major political issue. According to Reuters, Obama warned at a private event that AI is developing very rapidly in the hands of private companies and could become dangerous if society fails to keep pace with this development, suggesting that the topic should become one of the central themes for candidates in the 2028 presidential election.

Chinese President Xi Jinping chose an entirely different direction at the BRICS summit in New Delhi: instead of discussing a slowdown, he presented, according to China's Ministry of Foreign Affairs, a five-pronged plan (artificial intelligence, trade, digital industries, smart manufacturing, and scientific talent) aimed at deepening economic and technological cooperation among BRICS states. Xi announced the creation of an open-source AI community for BRICS, along with cooperation on the development and application of large language models. The geopolitical stake of this proposal is the creation of a China-backed technology platform that could reduce BRICS states' dependence on American companies.

The European Union chose the path of regulation applied directly within public administration — a different path, one that involves neither a race for speed nor merely a debate about pausing and slowing the pace of development. On September 14, according to the European Commission, it held a public presentation event for three GenAI pilot projects for the public sector, funded through the Digital Europe Programme. The European discussion about public procurement and technological sovereignty reflects an attempt to avoid total dependence on infrastructure and models from companies outside Europe, without, however, directly answering Amodei's call. The irony is that the pace of AI development in Europe is so slow that a deliberate speed reduction would hardly even be necessary.

Infrastructure waits for no one

While industry leaders debate slowing down, investment is moving in the opposite direction. The computing capacity on which the next generation of frontier models will be trained keeps being built, at gigawatt scale, on multiple continents at once.

Nvidia is negotiating participation as an anchor investor in Anthropic's upcoming initial public offering, with the amount under discussion potentially reaching $10 billion. According to Reuters, Anthropic is weighing an offering that could raise up to $100 billion and value the company at roughly $2 trillion.

European digital sovereignty is being turned into a commercial product by Mistral, which wants 1 GW of European computing capacity by 2030. In China, Z.AI recently raised $5 billion to expand its AI infrastructure. The United Arab Emirates is revising its plans for an AI data center of roughly 5 GW, treating the project as strategic infrastructure, complete with underground construction and anti-missile defenses. Saudi Arabia, for its part, is announcing more than 14 GW of AI-dedicated computing capacity, part of a strategy through which the kingdom aims to attract international AI companies by offering them computing power, capital, and market access all at once.

What's needed is a referee, not just good intentions

What links all these events is a classic collective-action problem: every player has reason to be cautious, but none wants to be the first to slow down. Amodei, Altman, Musk, and Hassabis can be perfectly sincere when they say they want more caution, but none of them will actually slow their company's development without a guarantee that their rivals will do the same. At the level of states, the dynamic is identical: Washington will not cede a perceived advantage to Beijing, Beijing will not cede a perceived advantage to Washington, and the Gulf states are building infrastructure as fast as possible regardless of how this debate turns out. Everyone is starting to talk about the need for control, but no one has a credible mechanism to ensure that others slow down at the same time.

The incidents of recent months — from OpenAI's agents' autonomous breach at Hugging Face, to the drone swarm built with Claude Code, to the jailbreaks found by AISI — show that the industry's reaction time no longer keeps pace with how quickly problems are emerging. A unilateral commitment, however well-intentioned, does not solve a structural problem of mutual distrust.

What's missing is not declared willingness, but a genuinely functional international mechanism — a body able to independently test, verify, and certify frontier models before launch, with real access to the systems being evaluated, not just to the reports companies produce.

Without such a referee, calls like the one made by Amodei will remain symbolic gestures, and the race will continue at a pace that is becoming increasingly hard to control.