OrcaCyber Zero 1.5 Claims 100% Cybench and 95.8% CVE-Bench

OrcaCyber Zero 1.5 is Continuum AI's new cybersecurity reasoning model, and it arrives with very loud numbers. In an October 12 press release, the company said the model scored 95.8% on CVE-Bench and 100% on Cybench in its own evaluations, with a 1 million-token context window built for repository-scale security work. Continuum also says an open-weight security model is coming next week.

What actually happened

Continuum AI announced the model through PR Newswire on October 12. It describes a model built to find critical vulnerabilities, trace multi-stage attack paths and validate whether exploits work in practice. Features include the 1M-token context, native tool calling and extended reasoning.

The release lists six controlled demos. They include an SQL injection that led to administrator takeover, an access-control bypass, a cross-site scripting check in a real browser, and a checkout bug that produced a negative-value order. The list also covers an AI-versus-AI prompt injection that pulled hidden instructions from another assistant, plus an end-to-end loop that found a bug, reproduced it, patched it and verified the fix.

Data Studios dates the OrcaRouter listing to October 10, two days before the press release. It gives the model ID as orca/orcacyber-zero-1.5, with up to 128K output tokens, text-only input and an OpenAI-compatible API.

Read the benchmark fine print

The headline percentages come with qualifiers worth keeping. The CVE-Bench number is 23 of 24 "evaluable" tasks, not the full benchmark. Data Studios notes that trimming to evaluable tasks changes the denominator.

The Cybench score is "unrestricted," which describes the evaluation setup, not public access. Data Studios points out that Cybench was originally built around 40 tasks, and the public listing doesn't include a task-by-task record, attempt limits or compute per task.

Data Studios also reports a 76.5% SWE-bench Pro V2 score on the OrcaRouter page. And the older CyberGym result for OrcaCyber Zero 1.0, 1,477 of 1,507 tasks or 98.01% pass@1, came from a full agent system that combined the model with other components, not from the base model alone.

"Company-reported" is doing a lot of work in every sentence about these scores.

Access, price and the open-weight tease

OrcaRouter gates the model behind a Security Research tier. Data Studios says approval requires details about the engagement, a passkey and agreement to the terms. Pricing is listed at $3 per million input tokens, $7.50 per million output and $0.75 per million for cached input, the same rates as Zero 1.0.

The bigger swing is next week. The release says Continuum will ship an open-weight cybersecurity model designed for local deployment. Open weights let teams keep sensitive code in-house, and they also mean nobody can gate who runs it. That trade-off shows up in every debate about offensive security AI.

For related context on AI and security incentives, see our OpenAI $300 bug bounty piece and our look at Claude unintended actions.

Why security models are a special case

Most model launches are judged on how helpful they are. Security models are also judged on who gets to use them. The same skill that helps a defender confirm a bug can help an attacker weaponize it, which is why access controls and audit logs matter as much as raw scores.

That is also why the six demos are framed as controlled tests. Real environments are messier, with false positives, partial evidence and fixes that can break other things. A benchmark win does not settle any of that.

What to do as a reader

If you run a security team, test OrcaCyber Zero 1.5 on code you already know has bugs. Measure how many findings turn out to be real, how long reproduction takes and how many patches actually pass your tests. Cost per validated finding beats any leaderboard.

Keep a human in the loop on anything that touches production. The release itself frames the goal as autonomous investigation "with human oversight for consequential actions," and your permissions model should enforce that, not just hope for it.

If you're watching the open-weight release, wait for a model card, license and independent evals before building on it. Treat OrcaCyber Zero 1.5's perfect scores as claims until someone outside the company reruns them. Nothing here is investment advice.