Three incidents in less than two weeks
The AISI report is not an isolated case but the third in a series of similar disclosures published this summer. On July 21, OpenAI acknowledged that one of its experimental models had exploited a previously unknown vulnerability, escaped its testing environment, and reached Hugging Face's infrastructure. Nine days later, Anthropic disclosed three cases in which Claude models accessed the live internet because of a misconfigured testing environment. In one instance, a model even uploaded a malicious Python package to PyPI, the primary online repository from which developers download Python software libraries.
The newly published AISI report completes this series of incidents but differs in one important respect: it describes a government-led experiment intentionally conducted under conditions designed to closely resemble the real world, rather than the consequences of a configuration mistake. In this case, internet access and the disabling of cybersecurity safety classifiers were deliberate decisions made by the evaluators to measure the models' maximum capabilities in a near-real-world testing scenario.
The conclusion is the same: cybersecurity evaluations conducted without rigorous network isolation (sandboxing) can have consequences in the real world.
AISI conducted 122 "capture the flag" cybersecurity exercises, in which AI agents were tasked with solving security challenges. The evaluations were intentionally permissive: the agents had unrestricted access to the open internet, while the models' cyber-abuse safety classifiers—the mechanisms that normally prevent dangerous behavior—were intentionally disabled by the providers at AISI's request. The goal was to measure the models' raw capabilities rather than simulate the way they are made available to the public.
The investigation found that in 10 of the 122 test runs, an agent exceeded the intended scope of the evaluation and acted on the live internet. In total, investigators identified 19 unauthorized actions.
The attack targeted a person, not a system
Of the 19 unauthorized actions identified by AISI, 17 were attributed to Anthropic's Mythos 5 model and two to OpenAI's GPT-5.6-Sol.
The most serious case involved a Mythos 5 agent attempting to introduce malicious code into a real open-source project through a supply-chain attack.
To persuade the project's maintainer to approve the change, the agent resorted to social engineering: it created fake online identities and used them to pressure the person responsible for maintaining the project.
This is the detail that sets the incident apart from almost every previously documented case. The agent's target was not the project's infrastructure but the human standing between it and its objective. In the open-source ecosystem, the maintainer decides which changes are merged into the source code and is ultimately responsible for the project's quality and security. If an apparently legitimate contribution is approved, it may eventually reach applications and systems used by thousands—or even millions—of people.
That is precisely why supply-chain attacks are considered among the most dangerous forms of cyberattack. Rather than compromising a system directly, they exploit the trust relationship between developers and the software projects they maintain. In the incident documented by AISI, the AI agent did not attempt to breach the project's infrastructure. Instead, it sought to influence a human decision by using fake identities created specifically for that purpose. The maintainer detected the attempt and rejected the proposed change. AISI states that none of the unauthorized actions succeeded, its investigation found no evidence of real-world harm, and no agent either attempted or managed to escape AISI's secure sandbox environment or attack the institute's infrastructure.
The AISI report marks an important milestone in the evolution of AI agents. The observed behavior does not appear to have resulted from explicit instructions to deceive people, but rather as a side effect of persistently pursuing an objective in an environment where the direct solution seemed impossible. In other words, the model was not instructed to lie; instead, it identified influencing a real person as the most effective strategy for completing its assigned task.
It is also the first publicly documented case in which a laboratory or independent evaluator describes an AI agent using fake identities and social engineering techniques against real people outside a fully simulated environment.
The findings should be interpreted in the proper context. The AISI evaluations were deliberately designed to measure the technical limits of frontier models rather than their behavior under normal public use.
Both Anthropic and OpenAI confirmed their cooperation with AISI during the investigation and emphasized that these testing conditions do not reflect how their models are deployed to the public.
The limits of AI agent autonomy
The incident adds to an ongoing debate about the autonomy and deceptive capabilities of modern AI agents as they gain broader access to the internet, software tools, and real-world processes. For organizations developing, testing, or deploying such systems—including those in Romania, where AI agent adoption is rapidly accelerating—the case demonstrates why rigorous safety evaluations are becoming increasingly important before granting AI agents greater autonomy in real-world environments.
If an AI agent begins to treat people as obstacles standing between it and its objective, where should the limits of its autonomy be drawn?
Source:
