: AI-resistant art
Point an autonomous LLM-powered security agent at a target and tell it to find vulnerabilities, and it will find vulnerabilities. It finds them every time. Point it at a hardened, freshly patched box with nothing actually wrong on it, and you still get a report back, formatted nicely, with seemingly suitably alarming impact ratings and everything.
[snaps back into reality]
Unfortunately, current models tend to lean toward coming up with analyses that sound plausible instead of just saying they didn't find any vulnerabilities, especially when the prompt or agent workflow kind of assumes there's something to find. As a result, they can produce reports that look convincing and well put together, even though the vulnerabilities they describe aren't actually real.
So you end up with a pile of findings and no quick way to know which ones are real because the actual bugs and the completely invented ones show up looking identical.
An example of a finding an agent produced not long ago:
It even provided a PoC python script that, according to the transcript, ran and came back with a Success message.
Success
Maybe it's genuine? Or maybe the endpoint returned a 500 for some unrelated reason and the model read some error in the response as "injection confirmed".
500
Maybe the script printed Success but nothing in it ever actually checked whether it actually succeeded. Maybe the whole thing's a convincing hallucination, stitched together from a few thousand security writeups the model saw during training.
You can't tell which of these happened by looking at the report because the report reads exactly the same either way.
It gets even worse when you look at where these findings are coming from. A lot of the time the first pass is done by some fast, cheap model because you want to scan as much as possible without burning a ton of money. The problem is those models are usually the ones that freak out the most and report everything like it’s a disaster. An Azure Tenant ID sitting in some client-side JavaScript is suddenly a “High severity sensitive information disclosure” finding, even though it’s basically useless on its own, and from what I’ve seen, these models are pretty bad at understanding the actual context of the app or what the business impact really is. They see something that looks like a vulnerability pattern, match the signature, and file it.
So the stage that produces the most findings is also the stage you should trust the least, which is a bad combination when the report from that stage is the thing that could end up on someone's desk.
The obvious response is to write a firmer prompt:
Only report vulnerabilities you've verified. Don't report false positives
(which never sticks anyway)
You're asking one agent, working in one train of thought, to be the eager hunter who turns up bugs and at the same time, be the hard-nosed critic who throws most of them out. Those two want different things but they're living in the same head, and the hunter tends to win because turning something up feels like getting somewhere, and admitting you were wrong feels like walking backwards.
One solution is to stop asking a single agent to do both. Split the work across roles that don't share a train of thought and, more to the point, don't share a motive.
The Prover just does what the agent was gonna do anyway. It pokes at the target, finds what looks like a bug, and throws together a claim with whatever evidence it has. SQL injection at /login, looks critical, here's the poc.py that triggers it. The Prover is allowed to be eager (that's kind of the point). Nobody's telling it to slow down or be careful. It just cranks out candidates and hands them off to something else that's actually skeptical.
/login
poc.py
That something is the Skeptic. The Prover can run cheap, because turning up candidates is the easy part. The Skeptic runs on a stronger model with a bigger reasoning budget. It takes the Prover's claim and the evidence and goes after them hard in an attempt to knock the thing down.
The Judge comes last. It doesn't actually do anything on its own. It just looks at what the Prover found and what the Skeptic pushed back on, compares everything against the same set of questions about evidence, reproducibility, impact, and confidence, then makes the call. It's the only role that's really there to say no when it needs to, and since it wasn't the one digging around for the bug in the first place, it doesn't really have a reason to want the bug to be real.
Written out, the instruction that drives this is blunt about the three rounds and about not letting the first one stand:
MODE: panel. Treat each finding as an adversarial match played in three rounds, and play all three in full: (1) PROVER Build the strongest possible live reproduction and state the case that the finding is REAL, with captured evidence. (2) SKEPTIC Argue as hard as you can that it's a FALSE POSITIVE. Reproduce the unauthenticated / baseline behavior, disable redirects, check that the "impact" content is truly route-specific and absent from the baseline, and surface every alternative explanation. (3) JUDGE Weigh both rounds against the evidence and rule. Do not let the Prover round go unchallenged.
Run it lighter and the same session plays all three parts in sequence. Run it heavier and each round is a separate model with its own context, which is the version I trust, for reasons the last section gets into.
The Skeptic's work is mostly about the baseline, and the baseline is the part a single eager agent almost always skips.
Take that SQL injection at /login for example. The Skeptic reproduces the exact request the Prover recorded, using the real wire values rather than a tidied-up version, and watches what actually comes back. Then it goes hunting for the boring explanation. It pulls the same page while logged out, to see whether the "leaked" content was ever protected in the first place. It turns off automatic redirect following, because a lot of authentication-bypass findings are really a 302 to a login page that the tooling quietly chased all the way to a 200.
302
200
For a claim that rests on something like a vulnerable dependency, it checks that the installed version is genuinely in the affected range instead of trusting a banner. For blind and out-of-band bugs, the kind where nothing visible comes back, it waits for a real callback to a server it controls rather than reading success into a delay or a hunch. The rule underneath all of it is easy to state and hard to live by. Actively try to make the finding fail. Only once you've honestly tried and failed to break it are you allowed to call it real.
Underneath the theatrics, the Skeptic's prompt is an unglamorous checklist it has to run before it's allowed to agree with anything:
Argue the other side. Actively try to make the finding FAIL. Does the "impact" content also appear in the logged-out baseline? Was a redirect being followed to a page that only looks protected? For a version or CVE claim, is the installed version actually in range? For a blind finding, did a real out-of-band callback fire, or are you inferring success from a delay? What is the most boring explanation that fits the same evidence? Only after you've genuinely tried to refute it may you confirm it. Evidence must be output you captured this session, never a paraphrase and never a reconstruction. If you can't capture it, the verdict is INCONCLUSIVE or NEEDS-MANUAL, never CONFIRMED on inference.
A finding is confirmed when the reproduction ran live and the captured evidence shows the stated impact. An actual record that should have been out of reach, not just a 200 that could literally mean anything. A finding is refuted when the conditions reproduce and the claim still falls over. The sensitive content turns out to sit in the logged-out baseline too, or the protected page was only reachable because a redirect was being followed, or the vulnerable version isn't in range, or the proof of concept just doesn't fire a second time. Those two are the clean outcomes.
A finding is inconclusive when the target has moved, the endpoint is gone, a WAF started blocking, or the evidence stays genuinely ambiguous after a real effort, and the Judge says exactly that instead of guessing. A finding is needs-manual when confirming it would take something the system can't or shouldn't do on its own like click a link in an email, tap a hardware key, or run the one destructive step that'd actually prove it. The verifier is read-only by design, so a bug that can only be confirmed by breaking something gets handed off with a written description of the safe manual check rather than a made-up result.
Two smaller disciplines fall out of this. Evidence has to be something the Skeptic actually captured this session, the real request and the real response, not a paraphrase and not a plausible reconstruction. If it can't be captured, the finding can't be confirmed, and it drops to inconclusive or needs-manual. And surviving the Skeptic doesn't lock in the original severity. A confirmed bug still gets re-rated, and plenty get marked down. An "admin data exposure" that turns out to require admin access to reach in the first place is real, technically, and also close to worthless, and the rating should say so.
The ruling itself is a structured object, not a paragraph of vibes. Each finding the Skeptic touches gets one written back onto it, which is what lets the report carry a per-finding badge and what forces the reasoning to be spelled out instead of implied:
{ "verdict": "CONFIRMED", // CONFIRMED | REFUTED | INCONCLUSIVE | NEEDS-MANUAL "confidence": "high", // high | medium | low "reproduced": true, "evidence": "", "reasoning": "why this verdict, including the refutation that was tried", "adjusted_severity": "medium", // re-rated; may sit below the original "artifacts": ["poc_verify_3_sqli_login.sh", "poc_verify_3_sqli_login.log"] }
The adjusted_severity sitting a notch under a CONFIRMED verdict is the re-rating in action. The bug is real and reproduced, and it still isn't the critical impact finding the first pass claimed.
adjusted_severity
CONFIRMED
When the ruling is done it rides back onto the original finding, so the report that started as a wall of undifferentiated claims now carries a verdict on each one. You can see at a glance which survived and which got struck down.
You could write all three of these behaviors into a single model as instructions. Discover first, then criticize, then judge, but it slides right back into the same tangle of competing motives that beat the firmer-prompt idea. The steps were never where the value sat. It sat in the Skeptic starting fresh, with a clean context and one plain instruction, which is to tear the claim down. It hasn't spent forty turns talking itself into believing the bug it just found. It reads a claim from someone it has no stake in, and its sole purpose in life is to find where that claim breaks.
The cleanest version of this gives each role its own process and its own context, so the Skeptic literally cannot see the Prover's private reasoning, only the claim and the evidence it chose to stand behind. A lighter version plays all three rounds inside one session, one after another, with the model told in plain terms to switch sides each round and to leave the Prover's case unchallenged at its peril. That still helps, because the adversarial framing does most of the work. The separate-context version is stronger for the obvious reason. A model that never watched you talk yourself into the bug is harder to talk into it.
This isn't free. It burns through tokens, and honestly it can burn through a lot, since the expensive parts run on the stronger model and the reject loop might have to spin a couple times before it finally lands on a verdict. For a lot of cases that's still cheaper than paying someone to sit there sorting the real issues from the junk by hand, but it's still a real cost. You have to decide if that's worth it based on how much a bad finding would actually cost you (or your business).
The Skeptic is only as good as its prompt. Give it a thin one and it will reproduce the finding, wave a hand at the baseline, and pass everything through, and now you have a chain of agents hopelessly nodding along to a hallucination inside a process that makes the nodding look like rigor. The Skeptic's prompt is the wall this whole thing leans on. Every concrete refutation step, pull the logged-out baseline, kill the redirects, check the version is in range, wait for the real callback, has to be spelled out, because "be skeptical" on its own buys you nothing.
And the Judge is only as calibrated as the model underneath it, so you should still keep an eye on it. A Judge that waves everything through is just a slower path to the same bad output. A Judge that rejects everything teaches the Prover to stop trying.
Build the thing so that being wrong has to survive a run-in with something whose only job is to prove you wrong. Telling a model "don't report false positives" was never going to hold. Giving a second, stronger model a reason to go catch them, a clean slate, and a verdict it has to sign, might have a chance.
insertchaos