A finding that reads “CVE-2023-xxxxx, version mismatch on port 443” gets closed within a day with a one-line comment: “not applicable, patched in the last maintenance window.” The argument that follows rarely hinges on severity. It hinges on proof. A security team that can attach the exact response, config value, or package output that makes the finding true wins that argument before it starts. A team that only has a scanner’s confidence score loses it, every time. This article covers what counts as evidence, when it is safe to collect, how to package it so a developer stops arguing, and how to stop doing it by hand.
What Counts as Evidence
Scanner output is a hypothesis, not proof. Evidence is the specific artifact that makes the hypothesis true for this asset, at this time. Five categories cover almost every case you will encounter.
- Service response. An HTTP response header, an error page fingerprint, or a TLS handshake detail that only appears in the vulnerable code path. A generic “Apache/2.4” banner is not evidence; a stack trace that leaks the exact module version because of an unhandled exception is.
- Config value. Reading the actual configuration, not inferring it from behavior. Confirming
PermitRootLogin yesin sshd_config, or connecting to a Redis instance and running INFO to confirm requirepass is unset, closes the argument that “we disabled that.” - Package state. The output of a package manager query against the live host, not the version string a proxy happens to forward.
- File hash. A SHA-256 of the actual binary or library on disk, matched against a known-vulnerable build. This matters when someone backports a security patch without bumping the version string, which happens more often on long-term-support distributions than most teams assume.
- Safe probe. A crafted, low-impact request that triggers behavior unique to the vulnerable code path, such as a timing delay from a blind injection point or a unique marker reflected unescaped in a response, without executing a payload that does real damage.
For Log4Shell-class findings, the evidence that ends a dispute is the SHA-256 of the running log4j-core jar plus the version string pulled from its MANIFEST.MF, not the fact that port 8080 responded to a JNDI callback. The callback proves reachability; the hash and manifest prove which build is actually loaded in memory.
Non-Destructive Validation Versus Destructive Proof-of-Exploit
Not every finding needs to be popped to be proven. Draw the line by asking one question: does this check write, delete, or execute anything on the target beyond what a normal client request would do? If yes, it is destructive, regardless of what the scanner vendor calls the module.
Non-destructive checks include passive banner and header inspection, direct config reads over an authenticated session, version diffing against advisory data, and canary-based confirmation, for example injecting a unique string into a parameter and observing whether it comes back unescaped to confirm a reflected XSS without ever rendering a script against a real user session. For a SQL injection candidate, a SLEEP(5) timing check combined with a blind extraction of version() is sufficient proof. You do not need to enumerate or drop a table to establish that the injection point is real.
Destructive proof-of-exploit, actually popping a reverse shell, running a credential-dumping tool, deleting a file to demonstrate path traversal, or exfiltrating real records, belongs in a narrower set of circumstances: written authorization that names the specific host and technique, defined rules of engagement with a rollback plan, and ideally execution against a staging clone or an isolated snapshot rather than the production system carrying live traffic. If the target is production and no rollback plan exists, default to the non-destructive path and accept that the proof will be slightly less dramatic. A developer who sees a timing delay and an extracted database version string closes the ticket just as fast as one who sees a full shell, and nobody has to explain a production incident afterward.
Authenticated Checks That Settle Version Disputes Once and For All
Most “false positive” disputes are actually authentication disputes. An unauthenticated scan reads a banner that a reverse proxy or WAF has rewritten, or that an admin quietly patched behind an unchanged version string. The fix is to stop arguing about the banner and query the host directly.
On Linux, that means an SSH session running dpkg -l | grep openssl or rpm -qa --last | grep openssl, comparing the exact version-release string, not just the major and minor numbers, against the vendor’s fixed-in field. On Windows, Get-Package -Name '*OpenSSL*' or a direct read of the relevant uninstall registry key does the same job. For containers, do not trust the image tag either: generate a software bill of materials from the running container with a tool like syft, and diff that list against the advisory’s affected package range. A tag of app:1.4.2 tells you nothing if someone rebuilt the image from a cached layer six months ago.
This is also where authenticated scanning earns its keep against a fleet of assets scanned through several different tools. When results are pulled together from multiple engines, the version each one reports can disagree simply because one queried a banner and another queried a package manager; reconciling that at the integration layer, the same layer that normalizes output from tools connected under scanner integrations, is what keeps one dispute from repeating itself on every asset the tool touches.
Assembling an Evidence Bundle a Developer Will Accept Without Arguing
A screenshot pasted into a ticket comment is not a bundle, it is a claim. A bundle a developer accepts on sight has a fixed structure, every time:
- Asset identifier tied to the CMDB or inventory record: hostname, IP, container ID.
- Timestamp and timezone of the check, so it can be correlated against a deploy log.
- The exact command or request/response pair executed, verbatim, not paraphrased.
- The tool and check ID that produced it, so the check is repeatable by anyone.
- Raw text capture of the observed value versus the value required for the CVE or misconfiguration to apply.
- The analyst identity or automation job ID, for chain of custody when the finding is disputed later.
Format matters as much as content. A structured JSON object attached directly to the finding record supports machine-to-machine tracking as the finding moves through its states, which is what a lifecycle system such as the one behind vulnerability lifecycle tracking depends on to know a fix actually addressed the original evidence rather than a related but different issue. A one-page PDF or rendered comment, generated from the same underlying record, is what the developer actually reads before closing the ticket. Generate both from one source of truth; never write the PDF by hand and the JSON separately, because they will drift. Tools built around a single evidence record, SITEY among them, render the developer-facing summary and the machine-readable bundle from the same underlying finding, which is what keeps a reopened ticket six months later pointing at the same proof instead of a stale screenshot nobody can find.
Handling the Honest “Cannot Be Validated” Outcome
Some findings cannot be safely confirmed, and pretending otherwise is worse than admitting it. A CVE in a third-party JavaScript widget served from a domain you do not control cannot be checked with an authenticated package query, because you have no host to authenticate to. A vulnerable function that only executes behind a feature flag currently set to off is real in the codebase but not reachable in the running system today. Forcing every finding into a binary confirmed-or-false-positive bucket creates a quiet incentive to fabricate confidence in one direction or the other.
The honest answer is a third state: unconfirmed, based on vendor advisory or static analysis, with an explicit confidence label distinct from validated-exploitable. Attach whatever partial evidence exists, the advisory text, the code path that references the vulnerable function, the reason a live check was not possible, and set a review date rather than closing the finding silently. Where risk exists without proof, document an interim compensating control, a WAF rule, a network segmentation change, an added monitoring alert, and time-box it. A compensating control without an expiration date has a way of becoming the permanent answer.
Automating Evidence Collection So It Happens on Every Finding, Not on Request
None of the above scales if it depends on an analyst remembering to run it. Past a few dozen hosts, manual evidence collection becomes the bottleneck that determines how many findings actually get proven before the next scan cycle buries them under new ones. The fix is to move evidence collection into the same pipeline that produces the finding, so it fires automatically rather than on request.
Concretely: when a scan reports a hit, a second, narrower, authenticated or safe-probe check runs immediately against that exact asset, and its raw output is attached to the finding record before a human ever opens the ticket. Platforms that automate this phase, such as SITEY, re-test the specific finding instead of trusting the patch command’s exit code, which matters because an exit code of zero tells you the patch script ran, not that the vulnerable version is actually gone from the host. The same automation that generates the initial evidence bundle during triage, the step handled by AI-driven triage in an eight-phase pipeline, is what later closes the loop: when a fix is deployed, the system re-runs the identical check that originally produced the evidence and only marks the finding closed under retest and closure if the same evidence now comes back negative. That symmetry, the fix is proven closed the same way the finding was proven open, is what actually ends the arguing, because there is no version of the ticket where a developer can plausibly claim the check was unfair.
About SITEY
SITEY is an autonomous vulnerability management platform. It discovers, validates, prioritizes, remediates and re-tests vulnerabilities through an eight-phase automated pipeline, unifying output from 17 integrated scanners. SITEY is self-hosted: it runs in your own infrastructure and your findings are stored there. Outbound connections are limited to licence activation and the optional services you enable, such as an AI provider, CVE enrichment and patch catalogues. Pricing is 599 USD per month or 5,999 USD for a perpetual lifetime license. See pricing or how the platform works.