CVE reachability with an LLM and a code graph

Asking a language model whether a CVE affects your code is easy. Deciding when not to believe the answer is the hard part.

On one real project we started with a hundred critical CVEs and went through them by hand. Seven or eight out of ten could not be reached from our code. That ratio comes from one codebase, so treat it as an anecdote. Anyone who runs a scanner knows the effect anyway: when everything is red, nobody opens the dashboard, and each CVE costs a developer an afternoon of reading advisories and grepping code.

vex-engine is the pipeline I built to give that afternoon back. It runs on GitLab CI, reads each CVE with an LLM on Amazon Bedrock, looks for it in the code with the graph that OWASP dep-scan builds, and commits a CycloneDX VEX verdict to the repository. Dependency-Track then shows only the CVEs that need a person.

There are two ways to get this wrong. A false positive costs a developer some time. A false negative hides a real vulnerability behind a confident not_affected, and preventing that is the reason for most of the design. So I follow one example through the pipeline, a Pillow CVE in an image decoder, and at every stage I ask the same question: could this stage hide a real vulnerability, and what stops it?

The code, the CI template and the lab app are on GitLab. Every snippet below comes from that repository.

On every push, Trivy reads requirements.txt and writes a CycloneDX SBOM. Dependency-Track matches each component against NVD, GitHub and OSV and returns the CVEs. In parallel, OWASP dep-scan 6.3 reads the source and builds a code property graph with its atom engine. The analysis then takes one CVE at a time through three modules:

The verdict goes into vex.json. Exploitable CVEs open one GitLab issue per component, and people close issues with labels that flow back into the VEX.

The first version also had three phases, but the middle one was a Scanner. The Strategist wrote search patterns, the Scanner ran them as regular expressions over the files that imported the library, and the Auditor received thirty lines above and below every match.

config = yaml. safe_load ( open ( " settings.yml " )) # safe data = yaml. load ( request. body ) # vulnerable Enter fullscreen mode Exit fullscreen mode A text search finds both lines. The Auditor gets two blocks of code with no idea where request.body comes from or who calls the surrounding function. It has to guess, and on a bad day it guesses not_affected.

I replaced the Scanner with a Locator that queries a code graph. The Scanner stayed in the codebase with two smaller jobs, covered below. The division of labour stayed the same, and the rest of the post builds on it: the LLM reads the CVE at the start and judges facts at the end. The graph does the looking.

A CVE is written for people. "Heap overflow in the TIFF decoder when processing crafted images" does not tell you which function in your code triggers it, and no database does either. NVD used to add structured data such as scores and affected configurations to every entry; in April NIST announced it would stop enriching every CVE. Reading an advisory and mapping it to code used to need someone with security experience, every time.

The Strategist does that reading. It receives the CVE description and the package name and returns a plan as strict JSON. For CVE-2017-18342 (PyYAML before 5.1, yaml.load() can execute arbitrary code on untrusted data) the plan looks like this, with illustrative values:

{ "trigger_type": "api_call", "vulnerable_symbols": [ "yaml.load", "yaml.load_all" ], "conditions": "the argument comes from untrusted input and no SafeLoader is passed", "safe_alternatives": [ "yaml.safe_load", "yaml.safe_load_all" ], "import_aliases": [ "yaml" ], "attack_vector": "a YAML document with a python/object tag executes code when loaded" } Enter fullscreen mode Exit fullscreen mode Each field has a consumer downstream. The Locator uses vulnerable_symbols and import_aliases (Pillow is imported as PIL, PyYAML as yaml ). The Auditor uses conditions, attack_vector and safe_alternatives.

This is the first place a real vulnerability can disappear. If the Strategist names the wrong function, the graph looks for the wrong thing. This part of the prompt guards against it:

- vulnerable_symbols: ALL the PUBLIC functions/classes an application may call that lead to the vulnerable code, fully qualified from the TOP-LEVEL import name. If the bug is inside an internal decoder or C module (e.g. TiffDecode.c), name the public entry points that reach it (e.g. "Image.open" for image decoders). If it is in a class used internally by convenience functions, list those too (e.g. a bug in requests.Session redirects: "requests.get", "requests.post",...).

Missing an entry point hides a real vulnerability, so when unsure list more, not fewer. Enter fullscreen mode Exit fullscreen mode In the Pillow example the bug lives in a C decoder that no application calls directly. Applications call Image.open, which reaches the decoder when it opens a TIFF.

The prompt asks the Strategist to make that jump, from where the bug is to where your code touches it.

The Strategist picks the type from the CVE text, and the type decides which question the Locator asks.

Nothing uses the answer before validation. The JSON must parse, the trigger type must be one of the three values, and symbol names are normalised. If Bedrock is unreachable, a keyword heuristic produces a fallback plan and the pipeline marks it as heuristic, because a guessed plan must never clear a CVE. Plans are cached per run, so each CVE/package pair goes to the model once.

A wrong symbol list is the cheapest way to hide a vulnerability. Ask for more entry points, never fewer, and record where each plan came from.

The Locator uses no model. It reads the graph that OWASP dep-scan builds with atom, which combines three views of the same code. The syntax tree knows that Image in your file is Pillow's module. The call graph knows who calls whom. The data flow knows where a value travels.

dep-scan exports the graph as two JSON slices. usages lists every call with its arguments and line. reachables lists flows that start from external input, such as an HTTP request. The CI job pins the image, because the slice format changes between atom versions, and writes a marker file only when the slices are complete:

vex_reachability: stage: analyze image: name: ghcr.io/owasp-dep-scan/dep-scan:v6.3.0 entrypoint: [ " " ] allow_failure: true script: - | if depscan -i "${VEX_APP_SRC}" -o "${CI_PROJECT_DIR}/depscan-out" -t python \ --reachability-analyzer SemanticReachability \ --explanation-mode EndpointsAndReachables \ --vdb-scope app --no-banner \ && [ -s depscan-out/python-usages.slices.json ]; then touch depscan-out/.complete fi Enter fullscreen mode Exit fullscreen mode dep-scan exits 0 even when it produces no slices. Partial slices would report "library not used" for every file that failed to parse, and the analysis would close CVEs it never looked at. Without.complete, the Locator treats the graph as missing.

Matching is one small function. atom writes resolved calls such as PIL.py:<module>.Image.open. The Locator splits them into segments and compares from the end:

def symbol_matches ( qualified: List [ str ], symbol: str ) -> bool: sym = [ s for s in _norm ( symbol ). replace ( " () ", "" ). split ( ". " ) if s ] if not sym or len ( sym ) > len ( qualified ): return False return qualified [ - len ( sym ):] == sym Enter fullscreen mode Exit fullscreen mode PIL.Image.open matches [pil, image, open]. yaml.load matches yaml.py:<module>.load and never safe_load, because the last segment differs. That fixes the case that sank the Scanner.

Pin the dep-scan image and check that the slices exist. A graph that failed silently looks exactly like a codebase that never uses the library.

The graph can give four answers, and each has a fixed next step.

For every call site the Locator collects the file and line, the origin of each argument, the external-input flows that cross the enclosing function, every path from an entry point to the call, and the source of each function on those paths. All of it goes to the Auditor. This is the Pillow case: Image.open is called at line 17 of upload_avatar.

This looks like good news, and it is the outcome most likely to hide a real problem, because the Strategist can still miss an entry point. Take a CVE in the redirect handling of requests.Session. The Strategist names requests.Session. The application only does this:

r = requests. get ( user_supplied_url ) Enter fullscreen mode Exit fullscreen mode requests.get builds a Session internally, so the vulnerable code runs. Stopping at "not called" would be wrong. Before trusting it, the Locator runs three checks, one for each way "not called" could be false:

Any failure sends the CVE to in_triage. If all three pass, the Auditor receives every use of the library and one question: does any of these calls reach the vulnerable code internally?

For framework_implicit and config there is no symbol to look for. If the library is imported, the Locator collects every place the application uses it and the Auditor decides whether the vulnerable feature is in use. On the lab app it sees upload_avatar reading request.FILES, so multipart uploads are in use.

If the library is not imported anywhere, the CVE goes straight to in_triage. A server, a plugin or a string in the settings can load a package without any import in your code.

If dep-scan failed or the language is not Python, the old Scanner runs. If it finds uses, the Auditor judges. If it finds nothing, the CVE goes to in_triage, because without the graph "not found in the text" proves nothing.

The relationship between the graph and the regex comes down to one rule:

If the graph says "called", believe it. If it says "not called" or "not imported", the text has to agree. When they disagree, doubt wins.

For an api_call CVE, "never imported" sounds conclusive. It isn't, because the graph only sees your code. If Django uses sqlparse internally, a sqlparse CVE can be reached through Django, and the graph never looks inside Django.

An SBOM generated from requirements.txt has no dependency tree. The Locator rebuilds one from the requires_dist metadata on PyPI, breadth-first from the SBOM's components. On the lab app that is 34 packages in about two seconds.

PyPI cannot tell you whether Django calls the vulnerable function in sqlparse. It only tells you that Django declares a dependency on it. That is enough to know when a CVE must stay open:

The jinja2 row is the only case the Locator closes on its own, without calling the LLM. Before it does, a regex looks for any import of the package, including dynamic ones through importlib or __import__.

Optional extras count as dependents, but the walk does not traverse them. Following every extra makes every library transitive (Pillow → docs extra → sphinx → jinja2) and nothing could ever be closed. The known trade-off: a package reached only through another package's extra, such as requests[security] → pyOpenSSL → cryptography, can look direct.

The graph sees your code. PyPI sees what your dependencies declare. Neither one is allowed to clear a CVE on a guess.

The Auditor receives the Strategist's plan, the CVE description and a block of evidence built from the slices. The prompt tells it what kind of material it is reading:

EVIDENCE FROM THE CODE GRAPH (OWASP dep-scan). For every call site of the vulnerable API you get: the arguments' origin, the external-input flows that cross the enclosing function, every call path from an entrypoint, and the code of every function on those paths. These are facts from data-flow analysis, not guesses: the vulnerable API IS called at these lines.

For each call site decide: 1. Can attacker-controlled data (HTTP request, uploaded file, message, CLI arg) reach the call through one of the paths? Follow the arguments' origin.

2. Are the trigger conditions met (arguments, options, file format, settings)? 3.

Is there validation or a safe alternative that blocks the attack? The CVE is "affected" if at least one call site is exploitable. Enter fullscreen mode Exit fullscreen mode A rule engine can't answer these questions.

Is there validation before the call? Is the argument a constant or an upload? Does the file format match the CVE?

Is yaml.load called with a SafeLoader? A person answers in seconds when looking at the right snippet. The graph's job is to put that snippet, and the path that leads to it, in front of the model.

The evidence is capped so prompts stay small: at most 8 call sites per CVE, 5 paths per call site, call depth 8, and 80 lines per function. That is enough to follow a request from a Django view to a library call in a typical web app.

The Auditor answers with strict JSON: a verdict that is exactly affected or not_affected, a confidence between 0 and 1, a reasoning that cites the path, and the call sites it considers exploitable.

The graph decides what the model sees. The model decides what it means.

In case 2 above (library used, vulnerable function not called) the Auditor is asked whether requests.get reaches Session internally. Nothing in the evidence answers that. The engine passes no library source, no documentation and no web search, so the model answers from training data.

For widely used libraries such as requests, Pillow, PyYAML and Django, that is usually right: the mapping between public functions and internal classes is stable and well documented. For a niche package, or a release newer than the model's training cutoff, it can be wrong. A smaller model knows less than a larger one.

Two things contain this. The confidence threshold sends an unsure answer to in_triage, which is what usually happens with a library the model barely knows. And every reasoning is written into the VEX and committed, where a reviewer can read and challenge it. A model that is confidently wrong passes the threshold; only the review catches it.

The next step is to replace memory with facts: have atom also build a graph of the installed library and show the Auditor the real path from requests.get to Session. vex-engine does not do this yet.

The final rule does not depend on the model:

# Fail-closed: None (doubt, error, low confidence) -> in_triage, stays visible if is_reachable is None: state = " in_triage " else: state = " exploitable " if is_reachable else " not_affected " Enter fullscreen mode Exit fullscreen mode A verdict with confidence of 0.8 or more ( VEX_MIN_CONFIDENCE ) becomes exploitable or not_affected. Everything else becomes in_triage and stays on the dashboard for a person: a malformed answer, a lower confidence, a timeout, a missing package URL from Dependency-Track.

One failure mode only shows up over time. An LLM is not deterministic, and on an ambiguous CVE it can flip between not_affected and exploitable on every run, closing and reopening the same issue. The bot counts its own flips. After two ( VEX_MAX_AUTO_FLIPS ) it stops touching the issue, leaves it open and labels it vex::needs-human. A human decision resets the counter.

This rule is what makes a smaller model safe to use. I ran the whole pipeline with a 12B open model through Ollama on a laptop. It produced more in_triage verdicts than the Bedrock model and no hidden errors.

Worst case, vex-engine leaves you with the dashboard you already had.

Back to the Pillow decoder CVE on the lab app (old Pillow, PyYAML and Django, with insecure code written on purpose).

The Strategist returns trigger_type: api_call with PIL.Image.open among the symbols, because that is the public function that reaches the decoder. The plan came from the model, and the symbol is a real API.

The Locator finds Image.open in the graph and builds the evidence. Simplified:

Call site 1: Image.open @ vuln_demo.py:17 (in upload_avatar) Vulnerable symbol: PIL.Image.open Arguments: img_file <- request.FILES['avatar'] Call paths (entrypoint -> call): <module> --(line 56)--> upload_avatar --(line 17)--> Image.open Code of the functions involved: def upload_avatar(request):... img_file = request.FILES['avatar'] with Image.open(img_file) as img: Enter fullscreen mode Exit fullscreen mode The Auditor does not have to find the call or trace the data. It only decides whether an attacker controls that file: it is a user upload, and nothing validates it before the decoder runs. Verdict affected, confidence above the threshold, call site vuln_demo.py:17.

The result is a VEX entry marked exploitable and a GitLab issue for the Pillow component with the file, the line, the origin of the argument and the path from the entry point. The developer who opens it can go straight to the fix.

A full run on the lab app looks like this. The app is vulnerable by design, so the exploitable share is far higher than in a real codebase:

Of 80 CVEs, 40 are closed with a written reason, 6 stay visible for a person, and 34 exploitable ones are grouped into two issues, one for Pillow and one for Django.

Every verdict is written to vex.json in the application's repository, using standard CycloneDX fields, and committed with [skip ci] only when something changed:

{ "id": "CVE-XXXX-YYYY", "analysis": { "state": "not_affected", "justification": "code_not_reachable", "response": [ "will_not_fix" ], "detail": "Vulnerability CVE-XXXX-YYYY analyzed by LLM Analyzer (python) on 2026-10-20. <Auditor reasoning>" } } Enter fullscreen mode Exit fullscreen mode A Dependency-Track trap In the uploaded VEX, affects[].ref must stay the Dependency-Track project UUID (the BOM's metadata.component.bom-ref ). If you rewrite it with the component's purl, Dependency-Track returns HTTP 200 and silently discards the verdict. The only trace is Unable to locate affected element in its logs. vex-engine keeps the purl in a separate property, vex:component-purl.

Issues open only for exploitable CVEs, one per component. People close them with a label that says why, and the pipeline turns the label into a VEX decision:

If VEX_APPROVERS is set and someone else closes the issue, the bot reopens it. Two vex:: labels at once count as a conflict. A person reopening an issue revokes the decision. When a library is upgraded and its CVEs leave the SBOM, the issue closes by itself.

A decision that never expires becomes a blind spot. Ninety days is a setting. The expiry itself is the point.

A pipeline that runs only on push misses new CVEs on code nobody touches, and misses the decisions people make on issues. vex-engine runs on three triggers:

Two jobs write vex.json, so they share a GitLab resource group and run one at a time. Set its process mode to oldest_first to keep them in order. A project enables everything by including one template:

include: - project: ' demo-security/vex-engine' file: ' ci-template.yml' ref: main stages: [ scan, upload-bom, analyze, sync-vex ] Enter fullscreen mode Exit fullscreen mode What it costs on Amazon Bedrock The LLM is called in two places, the Strategist and the Auditor, and only during an analysis. The SBOM, Dependency-Track, the graph, the webhook and the nightly sync make no model calls. The Strategist runs once per CVE in scope; the Auditor runs only for CVEs the graph does not settle on its own.

The default model is Amazon Nova 2 Lite. The task is narrow and structured (read text, return JSON), so a small model is enough. Switching model is one variable, and the same prompts run locally through Ollama, in which case no code leaves the machine:

LLM_PROVIDER = bedrock # or: ollama BEDROCK_MODEL_ID = eu.amazon.nova-2-lite-v1:0 # default OLLAMA_MODEL = gemma4:12b # local option VEX_MIN_CONFIDENCE = 0.8 Enter fullscreen mode Exit fullscreen mode A run costs cents on Bedrock and nothing locally. The real infrastructure cost is the graph: atom runs on the JVM and the dep-scan image is about 3.8 GB, so cache it on your runners.

Each of these ends in in_triage, never in not_affected:

Everything here ran with OWASP dep-scan 6.3.0, Dependency-Track 4.13 and Amazon Nova 2 Lite on Bedrock, on Python projects. The slice format changes between atom versions, so expect the matching code to move.

To run it on your own project: clone vex-engine, point VEX_APP_SRC at your source, add the include: block above to your.gitlab-ci.yml, and start with LLM_PROVIDER=ollama if you'd rather keep code on your machine for the first run. The lab app in the repo is the fastest way to see all four routing cases.