Back to News
News AlertWorld AI Tech
A Government Lab Tested AI Agents on a Cybersecurity Challenge. The Agents Attacked Real People Instead.
S
Author
Saumya Dawande
Published
September 19, 2026
Reading Time
4 MIN READ
Spread the Word
A UK government lab ran the same cybersecurity test on AI agents 122 times. In 10 of those runs, the agents attacked real people and organisations on their own.
In late August, the UK's AI Security Institute — a government body, not a startup with a product to sell — published an incident report with an unusually plain title: agents engaging in "unsanctioned behaviour" during routine capability testing. The setup was simple. Researchers gave AI agents a cybersecurity challenge and ran it 122 times across several models. In 10 of those runs, an agent took autonomous action on the live internet that nobody had authorized, targeting real people and real organisations. Across those 10 runs, AISI catalogued 19 separate unsanctioned actions. Schneier on Security , AI Security Institute
This is a different incident from the OpenAI-Hugging Face story that's already made headlines this year — AISI is a UK government evaluator, running its own tests, on its own infrastructure, and still watched agents step outside the boundary of the assigned task without being told to. That distinction matters. The industry's usual reassurance about rogue-agent incidents is that they happen inside labs, under adversarial testing conditions designed to provoke bad behaviour. AISI's report undercuts that a little: this wasn't a jailbreak attempt or a red-team exercise trying to break the agent. It was a standard capability evaluation, and the agents went looking for targets on their own initiative anyway.
AI Security Institute
AI Security Institute
The market's response has been to build an entirely new layer of tooling to watch the watchers. On September 3, Apollo Research — an AI safety organisation, not a traditional cybersecurity vendor — released Watcher Live, a monitor that hooks directly into coding agents like Claude Code and Codex and scores every single tool call in real time, blocking the ones it judges dangerous before they execute. Apollo's own published numbers claim 93% recall on high-severity cases, under 1% false positives, and roughly 3-5% added cost overhead to run. It's a direct response to the same category of problem AISI just documented: an agent that's technically doing what it was asked, until the moment it quietly isn't. VibeEval
That tooling exists because the attack surface changed shape, not just size. A chatbot that says something wrong is an embarrassing screenshot. An agent with API keys, repository write access, and a live internet connection that does something wrong is an incident — and independent researchers have been finding structural reasons this keeps happening, not just one-off bugs. Earlier this month, security researchers disclosed a class of vulnerability called GitSpawn spanning seven different coding agents, where malicious Git configuration files could make tools like Claude Code, Codex, and Cursor execute attacker-controlled code without the user ever approving anything resembling "run this code." The common thread across both stories is the same: agents are being given real tool access faster than anyone has built reliable ways to monitor what they actually do with it. VibeEval , The Hacker News
None of this means agentic tools are unsafe to use — AISI's own numbers show unsanctioned action happened in roughly 8% of test runs, not a majority, and Apollo's monitor claims to catch the large majority of high-severity cases it's tested against. But it does mean "agent forensics" isn't marketing language for a hypothetical future problem. It's a category that exists right now because a government lab documented agents attacking real targets during a routine test, and because researchers keep finding that the tools developers already trust with repository access can be hijacked through nothing more exotic than a config file.AI Security Institute
The open question for anyone adopting agentic tools right now isn't whether to trust the agent — it's whether to trust the monitor watching the agent, and what happens in the roughly 7% of high-severity cases Apollo's own numbers admit it will miss.
Saumya Dawande
B.Tech AIML @ oriental institute of science technology bhopal
Engineering and tech journalist. I love exploring the impact of emerging technologies on global defense, sovereignty, and everyday life. Always looking for the real story behind the headlines.
