AI Triage & Validation

AI Triage & Validation is the stage where a raw pile of scanner output becomes a ranked, deduplicated, evidence-backed work queue. Every finding gets a priority score from 0 to 100, a false-positive probability with the reasons that produced it, and a duplicate group if the same issue arrived from more than one source. In the AI analysis path, when the finding resolves to a registered endpoint agent, SITEY goes one step further: it generates a diagnostic script for that specific finding, runs it on the host, and feeds the real output back into the analysis, so the verdict rests on what the machine reports, not on what the scanner guessed.

The problem

A scan finishes and hands you a spreadsheet. Three scanners flagged the same CVE on the same box, so you have three tickets for one fix. A quarter of the “critical” rows are rated critical by the scanner but carry a CVSS of 4.1. Another slice has no CVE, no description and no suggested fix. Nobody can tell whether it is real without SSHing into the host and checking a version string by hand.

So the first day of every scan cycle goes to reading, comparing and chasing. An analyst messages a system administrator asking which patch level a server is on. The administrator replies two days later. The finding is dismissed as a false positive in a chat thread, which means that when the next scan re-reports it, the same conversation happens again from zero. Meanwhile the ordering of the queue is CVSS descending, which treats an internet-facing service on port 443 exactly like an internal one seen once on an unusual port.

How it works

1. Deterministic scoring runs first

   Before any language model is involved, each finding passes through a rule-based scorer. The priority score is a weighted sum of five components you can see broken out in the interface: CVSS (up to 40 points), risk level (20), how often the finding has been detected, on a logarithmic scale (15), the false-positive assessment (15), and whether the port is a commonly internet-facing one such as 443, 3389 or 22 (10). In parallel, a false-positive check looks for eight concrete signals, risk/CVSS mismatch, a high severity with no CVE assigned, a missing remediation section, a CVSS of 0 against a non-zero risk rating, a single-detection sighting, a very short generic name, an empty description: and returns a probability, the list of reasons, and a confidence level derived from how many signals fired. This layer costs nothing per finding and works with no API key configured.

2. Duplicates are grouped

   Findings are compared pairwise with a weighted similarity function: matching CVE carries the most weight, then token overlap in the vulnerability name, then matching port, risk level and scanner. Anything above the similarity threshold is grouped, and the group key and member count are stored on each record so the queue shows “this is one of six” rather than six separate rows.

3. Evidence is collected from the endpoint

   Where the finding resolves to a real registered agent, SITEY assembles host context for the analysis: operating system and build, installed software with versions, installed Windows KBs, pending-reboot state, running services, open ports, antivirus and firewall posture, free disk space, the outcome of previous patch jobs on that machine with a success ratio, and the other open findings on the same host. If a CVE is present, the record is enriched with NVD data, the EPSS exploitation-probability score, CWE, and a CISA KEV flag when the CVE is on the known-exploited list.

4. A diagnostic script proves or disproves the finding

   For agent-backed findings, the deep analysis path generates a read-only diagnostic script tailored to that vulnerability, checks it against a read-only command policy before anything is sent, dispatches it to the agent as its own job, and waits up to 90 seconds for the result. The raw output becomes the primary evidence in the prompt. If the script declares its own verdict, that declaration is explicitly treated as a claim to be checked against the raw output, not as proof. A contradiction has to be called out rather than accepted.

5. The decision is recorded and routed

   The model answers a fixed seven-part structure: false-positive verdict, confidence, technical rationale, attack-chain and exploitability analysis, concrete verification steps with expected results, recommended action and deadline, and an autonomous-closure assessment naming a closure method (Windows KB install, Linux package update, configuration fix, service disable, manual review, vendor contact). That assessment is parsed back into structured fields, and an operating-system sanity check overrides it when the method does not match the target, a Linux package update proposed for a Windows host is corrected and its confidence downgraded. Everything is written to a triage record: score, label, false-positive probability, the component breakdown, the full analysis text, model name, tokens used, duplicate group, batch id and the diagnostic output.

What you can do

Who it is for

Why it matters

The expensive part of vulnerability management is rarely the fix. It is deciding what deserves a fix.

Triage is where analyst hours disappear and where the queue quietly loses credibility: once a team learns that half the “criticals” are noise, they stop treating any of them as urgent.

Scoring findings deterministically and collapsing duplicates shrinks the actionable list before a human opens it. Collecting host evidence automatically removes the multi-day wait that sits between “the scanner said so” and “we checked”. And because every verdict is stored with its reasoning, a finding dismissed this month does not have to be re-argued next month. Findings that score 80 or above are labelled for action within 72 hours, which is what makes that deadline a working target rather than a slogan.

Works with

• Scanner Integrations, the source of the findings that arrive here.
• Vulnerability Lifecycle. Where triage outcomes become status changes and comments.
• AI Remediation, turns an accepted finding into a concrete fix plan.
• Autonomous Operations, runs the same analysis unattended under policy limits you set.

Frequently asked questions

Do I need an AI provider key to use this?

For the deep analysis, yes. You supply your own key for OpenAI or an OpenAI-compatible endpoint, configured inside your installation. That endpoint receives the finding text and the host context collected for the analysis, so it is worth choosing deliberately. The deterministic layer does not need one: priority scoring, false-positive signals, the score breakdown and duplicate grouping all run locally without any external call.

Will it close or dismiss findings without me?

It will never patch, remediate or resolve a finding on its own. Handing a finding to patch management creates a job in a pending-approval state, and approval is a separate, human step. Closure methods that are inherently manual, such as vendor contact, are never promoted to fully automatic. There are two things it does act on by itself, and you should know both: a read-only diagnostic script is dispatched to the agent during deep analysis without a separate approval, and a finding whose false-positive probability passes the threshold is set to false positive automatically, with a comment stating the probability written onto it so the decision is visible and reversible. In autonomous operation that threshold is part of the policy you configure.

How many findings can it process at once?

A single interactive request is capped at 500 findings server-side, and the interface splits a larger selection into chunks of 100 and shows progress as it goes. We do not publish a throughput figure for very large queues, so treat a multi-thousand selection as something to run in stages rather than in one sitting. For long runs there is a persistent queue, started from the AI assistant and capped at 200 findings per job, whose worker commits its position after each finding and picks up where it left off after a restart.

What do I actually get to show an auditor?

A stored record per triage: the score and its five components, the false-positive probability with the individual signals that produced it, the full analysis text, the model used, the timestamp, and the user who triggered it. For agent-backed findings analysed in the AI path, the diagnostic script output from the host is stored with it. Each vulnerability keeps its chain of triage records over time, so you can show how an assessment evolved rather than just its current state. These records are read inside the product; this module does not render them into a separate auditor-facing report file.

Triage is where a vulnerability programme is either won or lost. If your team is still sorting scanner output by hand, this is the module that pays for itself first. See what a full SITEY deployment costs on the pricing page. You install SITEY in your own infrastructure and your findings live in your database, and the deterministic layer runs entirely locally.