VULNpocalypse
InformationalVulnerabilityMonitoring

Endor Labs claims its AI SAST found 2.6x more real flaws than Claude

Endor Labs claims its AI SAST found 192 real vulnerabilities in an 8-project benchmark; treat it as a vendor claim and test on your own code.

Published
Exploitation
No evidence in the sources; the Endor Labs post does not report exploitation of any flaw it cites

What happened

According to Endor Labs, in a blog post it published on June 30, 2026 and updated on November 10, 2026, its own AI SAST product found 192 real vulnerabilities in a benchmark the company ran itself. Endor Labs says this is 2.6x what Claude (Opus 4.7) caught, 3.5x what Codex (GPT-5.5) caught and 2.4x what Semgrep OSS found (Endor Labs). This is a first-party claim from a vendor comparing its own product. The sources show no independent verification.

Endor Labs says it compared its tool with four traditional SAST tools (Semgrep OSS, OpenGrep, Bearer and CodeQL) and two frontier models prompted to find vulnerabilities directly. The corpus was 8 open-source projects in Java, Python, JavaScript, TypeScript, C# and Go. Endor Labs says its researchers built the ground truth by pooling every tool’s true positives, de-duplicating them by hand, adding issues found by manual review and verifying each one. It also says the AI-based tools were run multiple times and the aggregate reported (Endor Labs).

Other claims in the same post, all attributed to Endor Labs:

  • 63 of its findings were caught by no other tool, with the next highest at 18 (Claude Opus 4.7).
  • It detected 64 of the 106 CWE types in the benchmark, with the next highest at 36.
  • It had about 60% fewer false positives than Semgrep OSS and OpenGrep, which it says flagged over 460 each.
  • Its recall was 0.435 and its F1 was 0.465. Codex precision was 0.859 and Claude precision was 0.718, but their recall was near the bottom of the field.
  • It says CodeQL topped out at 0.211 recall.

The author states that Mythos-class models were not available for these results (Endor Labs).

Endor Labs also cites example findings from the benchmark:

  • A flaw in buffa, Anthropic’s Rust protobuf library. Endor Labs says untrusted wire data reaches an unbounded heap allocation through the default decode APIs, amplifying a small input into a roughly 22x memory increase that can OOM-kill the process. It says a CVE identifier exists for this flaw; the identifier and any official record are not confirmed here.
  • An insecure direct object reference in the product-image delete endpoint of Shopizer.
  • An authentication bypass in WebGoat.

Am I affected?

This is a benchmark claim, not a vulnerability in a product you run, so it matters only if you are weighing AI-based or rule-based SAST tools.

For the buffa finding, Endor Labs says the issue is reachable from the default decode APIs. Affected versions, fixed versions and any vendor advisory are not confirmed in the sources provided. The post points to a separate Endor Labs blog post for more detail, which is not among these sources. The sources do not report exploitation.

Not confirmed:

  • Independent replication of the benchmark results.
  • Tool versions, rule sets and configurations beyond the model names given.
  • How the results would transfer to your languages, frameworks or codebase size.

For context on method, NIST’s SATE V report says its goal was not to rank static analysis tools but to propose a methodology for assessing tool effectiveness using metrics such as recall and precision. It says results are examples and a basis for discussion (NIST). That report is separate from the Endor Labs benchmark.

What to do now

  1. Read Endor Labs’ methodology section and note that the vendor selected the corpus, built the ground truth and scored the tools.
  2. If you are evaluating SAST tools, run the candidates on a sample of your own repositories and compare true positives, false positives and recall against findings you have verified.
  3. Measure recall and precision, as NIST’s SATE V report does, and do not rely on a single headline multiple.
  4. If you depend on buffa, check the project’s own advisory channels for affected and fixed versions, since they are not confirmed in these sources.
  5. Revisit the comparison if independent results appear, as none are cited in the sources.

Sources