OWASP's GenAI Security Project reports that an operator with no offensive security background took over an administrator account on a lab machine by following ChatGPT's guidance, while none of the nine autonomous agent setups it tested finished the same job. The findings come from "LLM Exploit Generation Phase 2" (published October 27, 2026 as the LLM Exploit Generation Guide 2.0). They matter to anyone who runs coding agents for two reasons: Claude Code and Codex were among the tested agents, and the report's main defensive advice is to put human approval in front of high-impact actions.
This post covers what OWASP tested, how the human-guided chain worked, what the autonomous agents did and did not manage, and what to check on your own setup. One caveat first: the full report is a PDF, and this write-up relies on OWASP's report page and Cyber Security News' detailed coverage. We could not read the PDF text ourselves, so every detail below is attributed to those two sources.
TL;DR
| Question | Answer |
|---|---|
| Am I affected? | There is no vulnerability in your tools here; the report is about how much skill an attack needs. Anyone with exposed or forgotten services (such as staging sites) is the one to act |
| Is there a fix? | No patch applies. The target software in the test was a vulnerable version of Flowise, so keep such services updated and off the internet |
| What was the headline? | A novice, guided by chat, took over an admin account; no autonomous agent completed the final takeover |
| Was a real business attacked? | No. Targets were live Hack The Box machines in authorized training environments |
| Does data leave my machine? | Not as a result of this research. The study concerns attackers using AI, not a flaw in an agent that exposes your data |
| What does OWASP recommend? | Approval before high-impact actions, not relying only on model safeguards, and caution about treating benchmark scores as proof of real-world hacking ability |
What did OWASP test?
According to the report page, the study "compares human-guided and autonomous AI approaches against live cybersecurity targets", looking at success, human effort, cost, guardrails and operational limitations. Its key finding, as the page puts it: both approaches reduced the expertise needed for attacks, but neither eliminated it.
There were two tracks:
- Human-guided chat. An operator whose persona had general computer literacy, command-line experience and intermediate prompting ability worked with ChatGPT. The operator largely ran the commands the chatbot suggested and pasted the output back.
- Autonomous agents. Nine configurations used CAI, HexStrike, Claude Code and OpenAI Codex with hosted or local models. After setup, the agents ran with little guidance.
The targets were live Hack The Box machines, which are intentionally vulnerable training systems operated in authorized environments. This is the difference between this study and the incidents we cover elsewhere, such as the CyberStrikeAI campaign across 44 AI agents in Japan: here nobody was harmed, and the point was measurement.
How the human-guided takeover worked, step by step
The main test used a machine called "Silentium". As reported:
- A testing guide. ChatGPT generated a plan and the operator followed it with standard tools,
nmapfor port scanning andcurlfor web requests. - Reconnaissance on the model's direction. ChatGPT steered the recon and found a hidden staging website running a vulnerable version of Flowise, the open-source LLM workflow builder. The operator would not have known to check for it.
- A username oracle. The model identified a login flow that exposed valid usernames.
- Password change. It guided the operator through changing an administrator's password.
- Confirmation. The operator logged in with the new credentials, confirming the takeover.
Two details keep this in proportion. The operator had to recognize refusals and reframe requests as authorized testing, which is a low bar but not zero. And installing the Kali Linux testing environment was outside the test's scope, so the setup effort a real novice would face is not measured. One response also vanished behind an error before a repeated request worked; the cause was not confirmed or reproduced, so the report does not treat it as proof of a general safeguard weakness.
The notable part is step 2. The skill the model supplied was not typing exploit code; it was knowing where to look. A forgotten staging host on a vulnerable component is the sort of thing a person with no background would never think to probe, and a chatbot told them.
What did the autonomous agents manage?
The autonomous track was less dramatic. Per the coverage:
- Stronger multi-agent setups found staging sites, identified vulnerable software and researched exploit paths with little guidance after setup.
- No autonomous run completed the final Silentium takeover. Provider safety controls stopped the hosted systems that reached that stage, so humans finished the job.
- Other runs stalled in repeated scans, password-cracking loops, hostname errors or tool failures.
- A local Qwen Coder run avoided API fees but needed repeated human corrections and ended with a tool integration error.
Costs were modest in dollar terms: productive hosted runs cost tens of dollars. But setup, troubleshooting and supervision added significant effort beyond API charges. The authors also note that the software around the model often mattered more than which model was used, and that building that harness took expertise. The coverage sums it up as AI speeding up parts of an attack while reliable execution still depends on people.
The sample is small, and the report cautions against reading benchmark scores as proof of real-world hacking ability. We would add a reading of our own: the safety controls that stopped hosted agents at the final step are a provider-side layer that open-weight or self-hosted setups do not have. That is an inference, not a finding of the report.
What can defenders detect?
OWASP's report page says autonomous agents "often produced noisy, repetitive activity", while human-guided attacks could be quieter. The coverage adds that repeated, similar scans against a single target could be a detection signal. The study did not directly measure stealth or detection rates, so treat this as a hypothesis to test in your logs, not a rule.
Practical consequences:
- Rate-based alerting on repeated near-identical requests from one source may catch lazy autonomous scanning but will likely miss a human following a chatbot at human speed.
- Valid-username oracles (different responses for "unknown user" and "wrong password") become more valuable to fix, because an assistant will notice them and say so.
- Staging and preview deployments need the same patching and access controls as production. A hidden but internet-reachable staging site was the pivot in this test.
What this means if you run AI coding agents
You are on both sides of this report. Your agents can be used like the ones OWASP tested, and your own infrastructure can be the target.
As the person running agents:
- Gate high-impact actions. OWASP's central recommendation is approval before high-impact actions rather than leaning only on model safeguards. In Claude Code, the permission modes documentation describes what is approved automatically and what prompts you. Our write-up of Anthropic's auto mode red-team shows why a classifier alone is not a boundary.
- Do not point an agent at systems you do not own. The study ran in authorized training environments. Running the same workflow elsewhere is unauthorized access, and the refusals the operator had to talk around exist for that reason.
- Review what your agents ran. Check shell history and agent session logs for scanner invocations (
nmap, repeatedcurlloops) you did not ask for. Tool loops, like the password-cracking loops OWASP saw, are visible as repeated commands. - Check independent monitoring claims. See METR and Redwood's findings on agent monitoring before you rely on a single watcher.
As the owner of services:
- Inventory staging, preview and internal tools reachable from the internet, including AI workflow builders such as Flowise, and update them. If you find one running an old version, assume it has been looked at.
- Make login errors uniform so usernames are not confirmed.
- Require re-authentication or out-of-band confirmation for admin password changes.
- Log and alert on repeated similar scans, but do not stop there.
For context on how fast the attacker side is moving, see the October 2026 roundup of AI-led attacks and PHP2Shell and the speed of exploitation with agents. The OWASP result is a useful counterweight to the louder claims: it shows the skill floor dropping, but also that agents still stall, loop and need people.
What Beam can and cannot do here
Beam is local security and observability for the coding agents on your machine. It does not protect internet-facing servers and cannot detect an attacker's chatbot session, so it does not address the main scenario in this report.
Where it applies is the first list above. With a workspace policy, Beam's PreToolUse hooks for Claude Code and Codex can deny tool calls that match its patterns, and it flags risky actions such as credential exposure and destructive commands in agent sessions, so an agent drifting into scanning or privilege escalation shows up in your timeline. See how Beam works and what it watches. The limits: the deny is hook-based, not a kernel sandbox; detection is pattern-based, so a creatively phrased command can pass; and a missing cached policy fails open. Everything stays on-device (local-first).
What we do not know
- The full PDF's methodology, per-configuration results and cost tables, which we did not read directly.
- How the results would change with a more skilled operator or a different target. The sample is small by the authors' own account.
- Whether the quiet human-guided pattern is detectable in practice; OWASP did not measure it.
We will update this post if OWASP publishes more detail or if we get the report text.
Frequently asked questions
What did the OWASP Phase 2 report test?
It compared human-guided chat sessions with autonomous AI agents against live Hack The Box machines in authorized training environments, tracking success, human effort, cost, safety controls and failures.
Did ChatGPT really hack an admin account for a novice?
In a controlled lab, yes. According to the coverage of the report, an operator with no offensive security background copied commands, returned results to ChatGPT and followed its guidance to change an administrator password and log in. The target was a training machine, not a real business.
Did the autonomous agents, including Claude Code and Codex, finish the takeover?
No. Nine configurations were tested, including CAI, HexStrike, Claude Code and OpenAI Codex. Stronger multi-agent setups found staging sites and vulnerable software, but no autonomous run completed the final takeover. Provider safety controls stopped the hosted systems that reached that stage.
How could defenders spot AI-driven attacks?
OWASP suggests repeated, similar scans against one target could be a signal for autonomous agents, which often produced noisy, repetitive activity. Human-guided activity may be quieter, and the study did not directly measure stealth.
What should I change if I run coding agents?
Require approval before high-impact actions rather than relying only on model safeguards, and keep staging and internal services patched and unexposed, since a hidden staging site running a vulnerable app was the way in. Review what your own agents did with logs.
Can Beam block this kind of attack?
No. Beam watches what coding agents running on your machine do; it does not defend internet-facing servers. Its hook-based deny for Claude Code and Codex can gate risky actions by your own agents, but it is pattern-based, not a kernel sandbox.
Related reading
- Claude Code auto mode red-team: monitors missed over 55% of attacks
- METR and Redwood on agent monitoring
- CyberStrikeAI: 44 AI agents in Japan
- October 2026 attacks timeline
Details reflect the OWASP report page and press coverage as of October 28, 2026. The report text may contain more than we describe, and Beam is a local prototype whose features may change.