Ask five analysts on the same team to define “false positive” and you will likely get five different answers, which is the main reason false positive rate numbers rarely survive being compared across teams, tools, or quarters. Before the metric means anything, it needs a fixed definition, a sample analysts can actually produce without building extra tooling, and a reporting format that cannot be quietly improved by suppressing real findings instead of fixing anything. This covers all three, plus the sampling mistake that makes almost every vendor pilot look better than the same tool will perform once it is pointed at production.
Separate false positive from disputed, not applicable, and unexploitable
Most of the inflation in a reported false positive rate comes from folding several distinct dispositions into one bucket. They need separate codes and separate rules for who can assign them:
| Disposition | Definition | Who can set it |
|---|---|---|
| False positive | The scanner reported a condition that does not exist on the asset, for example a version banner misread after a load balancer terminates TLS one hop before the actual target | Analyst, with evidence attached |
| Disputed | Analyst and asset owner disagree on applicability or severity and verification is still open | Either party, pending review, never counted as closed |
| Not applicable | The vulnerable condition exists but the specific code path or function is confirmed unreachable | Analyst plus documented proof, such as a static call-graph excerpt |
| Compensating control | The vulnerability is real and reachable, but a control absorbs the risk to an accepted level, for example an unpatched SMBv1 host isolated behind a firewall rule | Risk owner, time boxed |
| Duplicate | The same underlying weakness reported twice, often by two different scanners against the same package or port | Automatic matching, or analyst override |
Only the first row belongs in the false positive count. Folding disputed, not applicable, or duplicate findings into “not a real finding” makes the rate look better while hiding work that still needs to happen, such as documenting a compensating control or resolving a disagreement with an asset owner. This is state that a vulnerability lifecycle should track as distinct statuses rather than collapsing everything into a single closed flag the moment a ticket stops being actively worked.
Build a labeled sample analysts can actually produce
A false positive rate is only as trustworthy as the sample behind it. Re-reading every closed finding from the last quarter by hand does not scale past a few hundred assets, so fix a sampling method and keep it constant period over period:
- Stratify by scanner and by plugin family before sampling, then draw randomly within each stratum, so one noisy plugin cannot dominate the overall number.
- Set a floor of 30 to 50 closed findings per stratum per quarter. Below that, a single disputed ticket swings the rate by several points and the figure stops being comparable across periods.
- Route roughly one in ten sampled findings to a second reviewer who did not set the original disposition. Disagreement above one in ten means the underlying dispositions need review before the rate gets published anywhere.
- Fix the sample window to a calendar period, such as the quarter, instead of a rolling window that can be nudged after the fact to land on a target number.
Analysts should be able to pull this sample from whatever system already records finding state. If producing it requires a side spreadsheet that only gets maintained for one quarter, the measurement will quietly stop happening by the second.
The formula, and the sampling trap that flatters every vendor demo
The formula itself is not complicated: false positive rate equals false positives divided by the sum of false positives and true positives, calculated only over findings that have reached a verified, closed disposition. Two mistakes break it in practice.
The first is using total findings raised as the denominator instead of closed and verified findings. A finding sitting untriaged in the queue is not yet a false positive or a true positive, it is unverified, and counting it as either pulls the rate in whatever direction the backlog happens to be leaning that week.
The second is the sampling trap behind almost every impressive demo number: a curated evaluation environment, a small lab, a set of patched golden images, a synthetic vulnerable application, has a narrow and well-understood attack surface, so nearly everything a scanner reports there is real. Production looks nothing like that. It has legacy software, custom internal applications, proxies that terminate TLS before the host the scanner thinks it is probing, and default-credential checks running against homegrown login pages that were never built with common defaults in mind. A scanner that shows a low single-digit false positive rate against a demo range can show a materially higher rate against a mixed production fleet, for reasons that have nothing to do with its detection logic and everything to do with what it was pointed at. Treat any number a vendor quotes without naming the environment it came from as a hypothesis to test, not a fact about your own fleet.
Break the rate down by scanner, plugin family, and asset class
A single blended false positive rate hides exactly where the noise is coming from. Split it across at least three dimensions before acting on it:
- Scanner: engines that infer a vulnerability from a version banner or service fingerprint tend to run hotter than ones that confirm a specific file hash, package version, or configuration value directly.
- Plugin family: default-credential and cipher-suite checks tend to run hot against custom applications and internal proxies, while checks tied to a specific CVE and package version tend to run cooler once the package manager is queried directly instead of the banner being inferred.
- Asset class: cloud workloads sitting behind a load balancer or WAF produce a different false positive pattern than on-premises hosts scanned directly, because the scanner may be probing the front door rather than the asset it believes it is reaching.
Once the rate is broken down this way, tuning effort goes where the noise actually lives instead of being spread evenly across tools that do not have the same problem. Getting output from a mix of network, web application, container, and cloud posture scanners into one normalized taxonomy, the kind of consolidation that scanner integration layers handle across a fleet of tools such as the 17 SITEY ships with, is what makes a breakdown like this possible without hand-building a mapping table for every new tool pairing that gets added later.
Reduce the rate without quietly suppressing true findings
Every false-positive-reduction initiative has a shadow option available to it: stop reporting the finding types that argue with people, and the rate drops without anything getting more secure. A few concrete rules keep that from happening unnoticed:
- Require evidence before a false positive or not-applicable disposition is accepted, not a text justification alone. For not applicable, that means a reachability trace, a config export showing the vulnerable function is never wired to input, or a network diagram excerpt showing the segment is unreachable, attached directly to the ticket.
- Scope plugin suppressions to the specific asset group or software version where the pattern was confirmed, never to the plugin globally. A TLS check that misfires against one proxy’s specific termination setup is not evidence the check is wrong everywhere else in the fleet.
- Put an expiration date on every suppression, tied to the next full scan cycle or 90 days, whichever comes first. Compensating controls get reconfigured, firewall rules get loosened for a migration and never tightened back, and a suppression with no expiry eventually hides a real exposure indefinitely.
- Re-test against the live asset after a patch or configuration change lands, rather than closing the ticket on the strength of a deployment log or a patch command’s reported exit code. A remediation step reporting success and the underlying vulnerability actually being gone are not always the same event. Platforms built around an explicit retest step, such as SITEY, re-run the original check against that specific finding once a patch is applied instead of trusting the patch job’s own report of success, which is exactly the gap a status-code check alone cannot see.
Report the number so leadership cannot be gamed by it
A false positive rate presented on its own, especially one that is falling quarter over quarter, tells a manager nothing about whether it fell because tuning genuinely improved or because inconvenient findings quietly stopped being counted. Give it enough surrounding context that gaming becomes visible:
- Publish the rate next to its sample size and total closed volume for the period. A rate computed from 40 findings should never be presented with the same confidence as one computed from 400.
- Include the second-reviewer disagreement rate from the sampling process. A 2 percent false positive rate sitting on top of 25 percent reviewer disagreement is a measurement problem being reported as a tuning success.
- Report per-scanner and per-asset-class breakdowns alongside the blended figure, so leadership can see whether an improvement came from broad tuning or from one scanner’s gain offsetting another scanner getting worse.
- Attach an audit trail: who set each disposition, what evidence was linked, and when any suppression expires. A record that ties every disputed or suppressed finding back to a named reviewer and a timestamp, produced as a side effect of ordinary triage rather than assembled specially for a board meeting, is what turns the rate from a claim into something an auditor can walk through line by line.
None of this requires new tooling to start. A shared set of disposition codes, a fixed quarterly sample, and a second reviewer will get a small team a defensible number within one cycle. What it requires instead is treating the false positive rate as a measurement to be audited, not a headline that can be improved by redefinition rather than by work.
About SITEY
SITEY is an autonomous vulnerability management platform. It discovers, validates, prioritizes, remediates and re-tests vulnerabilities through an eight-phase automated pipeline, unifying output from 17 integrated scanners. SITEY is self-hosted: it runs in your own infrastructure and your findings are stored there. Outbound connections are limited to licence activation and the optional services you enable, such as an AI provider, CVE enrichment and patch catalogues. Pricing is 599 USD per month or 5,999 USD for a perpetual lifetime license. See pricing or how the platform works.