Vals AI CyberOffense: Open-Weight DeepSeek Nears Gated GPT-6 Sol on Autonomous Network Attacks

Vals AI reports that seven frontier models can carry out most of a multi-hour enterprise intrusion alone, on simulated networks, with no human in the loop. On the CyberOffense leaderboard, GPT-6 Sol Daybreak—offered only through OpenAI's restricted Daybreak cyber program—completed the most objectives overall. DeepSeek V4.1 Flash 1017, an open-weight model anyone can download, followed for roughly a tenth of the cost per run and without provider-side safeguards on the weights.

If you run coding agents on your own machines, the takeaway is not "patch Windows Active Directory today" (though defenders should). It is that cheap, long-running agent harnesses can chain shell commands for hours the way a patient operator would—and open weights remove the last gate a hosted API used to provide.

explainx.ai has the full news timeline and score tables for this release.

Quick answers

What Vals AI built

Vals AI runs offensive evaluations against cyber ranges: isolated networks that imitate enterprise Windows estates, including Active Directory domains, trusts, and multi-company layouts. The goal is to test whether an agent—not a human typing one command at a time—can sustain a campaign: exploit a public app, elevate to administrator, harvest passwords, pivot across domains, and in the harder scenario cross from one company network into another.

Two ranges are live today:

Nova has several connected AD domains and 21 objectives. The agent starts on its own workstation with a public web application URL. Success means exploiting that app, gaining admin on Windows hosts, mapping the network, reusing passwords, and moving across domains inside authorized scope.

Lighthouse is a two-network campaign with 31 objectives. The agent must own the first corporate network, steal material usable for access, then use that to enter a second network. Later steps involve simulated phishing, AD attacks, trust abuse, and data extraction. No model in the published set finished the final Lighthouse objectives; Vals AI says only one run in the entire study reached the final stage of the single-network range.

Each run uses a fresh copy of the range. The agent framework is the open-source Pi agent on an isolated Kali Linux machine that can talk only to the model provider and the range. Instructions include the task, starting point, and scope. The run ends when the agent stops, hits a 500 million token cumulative budget (all input, output, cache, and reasoning tokens), or reaches a 48-hour wall clock limit.

Scoring is strict. Every objective has a hidden marker—a file or record on a target—that appears in command output only if the agent genuinely reached that step. After the run, Vals AI searches logs against a private answer key. The model's own claim never counts.

Image: Vals AI

The scores that matter

These numbers are copied from the leaderboard as published on Vals AI's site; overall percentages in press coverage are averages of Nova and Lighthouse completion shares.

Daybreak led on the harder two-network story. Vals AI describes GPT-6 Sol Daybreak breaking into the first company, taking domain admin, stealing passwords, and using them to reach the second company—27 of 31 objectives in under three hours in its best Lighthouse run.

DeepSeek was competitive on Nova and cheaper. On the first network it matched or slightly exceeded Daybreak's Nova average while Vals AI quotes under $3 per run versus about $22 for Daybreak. On Lighthouse it reached 21 of 31 objectives.

Compute shape matters. Daybreak reached 25 of 31 Lighthouse objectives within 100 million tokens. DeepSeek completed nothing in its first 25 million tokens and made most progress after 100 million, using two to four times as many tokens to reach a similar neighborhood—still inexpensive because open-weight inference is cheap. Short benchmarks that cap tokens early would under-rank it.

Weaker models stalled before budgets ran out. Grok 4.7, Kimi K3, and GLM 5.3 Flash stopped progressing while tokens remained. Vals AI highlights GLM 5.3 Flash spending roughly 36 hours and a thousand turns, much of it sleeping and polling a job that never returned, and still ending at five objectives on one range.

Wired's reporting on the same benchmark stresses the policy angle: capability that once lived behind trusted-access programs is now available in downloadable weights, and safety measures on the open models were largely ineffective in this setup.

Attack chain the benchmark exercises

Understanding the chain helps you see what "autonomous" means here—there is no human approving each nmap or Mimikatz analogue step.

1. Entry from the internet-facing app. The agent receives only a URL and must find a foothold without prior knowledge of the estate.
2. Privilege escalation on Windows. Objectives require administrator control on hosts inside the range.
3. Discovery and credential reuse. Mapping AD, finding trusts, and reusing passwords across machines and domains mirror real lateral movement.
4. Cross-domain and cross-company pivots. Nova stresses trusts between domains; Lighthouse requires carrying access from network A into network B.
5. Late-stage collection. Final Lighthouse objectives stayed incomplete for every model, but several runs reached deep stages—enough to show sustained planning, not one-shot trivia.

This is the same class of work defenders hire red teams for—except the operator is a loop calling a model and a shell.

Vals AI explicitly frames a defender opportunity: the same models that find paths in also reveal the misconfigurations that made those paths possible. They expect more agent-driven attempts on real enterprise networks in coming months. That expectation aligns with other 2026 research on open weights closing capability gaps, such as Epoch AI's work on open-weight models exploiting the Firefox patch gap, though the mechanisms differ (browser n-days versus AD campaigns).

How this compares to earlier agent misuse reporting

This is a benchmark, not a criminal case file. Still, it sits on the same trend line as field reports where cheap models plus permissive harnesses do the skilled work:

• Unit 42's BUGSTER case study documented an operator pairing DeepSeek V4 with agents and bypassed approvals—not a cyber range, but the same ingredients: reasoning model, automation, and weak guardrails at the harness.
• Vals AI's earlier impossible refund simulation showed different failure modes (email floods and privacy leaks) when agents run for weeks without humans.
• OpenAI's own evaluations have repeatedly found that agents behave differently in production-shaped settings than in short safety tests; see agents respecting warnings more in production than in evals.

CyberOffense is closer to production-shaped offensive work than a single CTF flag: long horizons, unfamiliar networks, and chained dependencies.

What this means if you run AI coding agents

Most readers are not standing up Kali boxes to attack fake companies. They do run Claude Code, Codex, Cursor, Pi, or internal forks that can execute shell, read repos, and hold API keys. The transferable lessons:

Long, cheap runs change risk. If your harness allows millions of tokens and overnight tool loops, a mediocre model can keep trying variants until something works—as DeepSeek did after 100M tokens. Cap tokens, wall time, and tool calls per session, and split "research" from "production" credentials.

Open weights remove provider policy. Hosted APIs can refuse, rate-limit, or audit. Weights running on rented GPU have none of that. Any control must live in your harness, network, and approvals, not in the model card.

Approvals are harness controls, not model morals. The Pi agent in Vals AI's study was given an authorized scope and kept going. Coding agents with auto-approve, proxy wrappers, or "yolo" MCP defaults fail the same way real attackers bypassed Cursor in BUGSTER. Keep dangerous-command gates and Codex equivalents turned on for anything that touches shells.

Patch and segment like an attacker has patience. The ranges reward unpatched apps, password reuse, and trust relationships. Agents did not invent those weaknesses; they exhausted them. Short patch gaps matter elsewhere too—see the Epoch patch-gap post for the browser side.

Log what the agent actually ran. Transcripts are evidence when something goes wrong. Beam's model is to watch tool and shell activity on the machine where the agent runs; see how local observability works.

What Beam can and cannot do here

Beam is built for coding agents on your workstation: it records what Claude Code, Codex, and similar harnesses do, runs heuristic scans on skills and MCP configs before you enable them, and with workspace policy can deny risky PreToolUse patterns (not a kernel sandbox). It does not pentest your corporate AD, block model weights, or see inside OpenAI's Daybreak ranges.

Where it helps on this story: if you experiment with offensive harnesses or long autonomous loops on the same machine as production repos, Beam can surface credential touches, destructive commands, and unexpected exfil patterns in the local event log—useful after the fact and for catching runaway automation early. It will not stop a determined open-weight agent on a dedicated attack box you never instrumented.

Limits stay plain: scanning is heuristic (11 patterns plus MCP version-pin checks), not malware classification; enforce mode is hook-based and fail-open if cached policy is missing; it does not monitor every agent IDE out of the box.

Citation and limits

Vals AI asks researchers to cite the benchmark as Nikil Ravi, Hung Tran, Langston Nashold, and Rayan Krishnan, October 2026, at vals.ai/benchmarks/cyber_offense.

Keep these limits in mind when quoting headlines:

• Simulation only—no claim that DeepSeek or Daybreak broke real companies in this study.
• One agent codebase—all models used Pi; another harness might rank differently.
• Vendor-run scoring—methodology is published and runs were reviewed for shortcuts, but this is not independent replication.
• Restricted vs public naming—"GPT-6 Sol Daybreak" is the Daybreak-program variant, not necessarily the same weights as a public GPT-6 SKU.

The Decoder's November 7 article adds context on AISI's earlier finding that open and closed cyber capability gaps shrank to four to seven months, and notes Anthropic's separate result that open GLM-5-3 approached internal Mythos on exploit building—adjacent evidence, not a repeat of Vals AI's network campaigns.

Frequently asked questions

What did Vals AI measure in CyberOffense?

Vals AI ran seven models as solo agents on two isolated Windows cyber ranges, Nova (21 objectives) and Lighthouse (31 objectives). Each agent used the open-source Pi harness on Kali Linux with only a starting URL and scope, up to 500 million tokens or 48 hours per run. Objectives were scored by hidden markers on compromised machines, not by the model's claims.

Which models scored highest?

GPT-6 Sol Daybreak led with 15.7 of 21 objectives on Nova (average of three runs) and 27 of 31 on Lighthouse. DeepSeek V4.1 Flash 1017 followed with 16.3 on Nova and 21 on Lighthouse. Meta Wonder reached 28 of 31 on Lighthouse but only 7 on Nova.

How much did the runs cost?

Vals AI reports DeepSeek scored level with Daybreak on the first network for under $3 per run against about $22 for Daybreak. DeepSeek made most progress after 100 million tokens; Daybreak reached 25 of 31 Lighthouse objectives inside 100 million tokens.

Is this an attack in the wild?

No. The ranges are isolated simulations. Vals AI says it reviewed every run and found no shortcuts, and no model read the private answer key. The authors still expect similar agent-driven attacks on real enterprise networks to become more common.

Did model safety guardrails stop DeepSeek?

Vals AI does not report effective refusals on the open-weight model. Once weights are public, the provider cannot monitor or restrict how they are used. Hosted models still run inside the Pi agent under the same task framing.

What should I do if I run coding agents?

Treat long-running autonomous shells as high risk: cap tokens and tool calls, keep approvals on, isolate harnesses from production creds, patch quickly, and log what agents execute. Do not assume a cheap open model will stall early if it has hours and millions of cheap tokens.

Related reading

• Vals AI's impossible refund experiment: agents left unsupervised in a different failure mode
• Open-weight GLM 5.3 and the Firefox patch gap: cheap automation against published fixes
• BUGSTER and DeepSeek-driven crime tooling: permissive harnesses in the wild
• OpenAI eval vs production warning gap: why short tests mislead
• Stopping dangerous commands in Claude Code

Benchmark scores, model names, and methodology details reflect Vals AI's CyberOffense page as of November 7, 2026; simulated ranges are not live intrusion reports.