filtering-unsupported-llm-generated-vulnerability-reports-security-triage-statistical-machine-textFor filtering unsupported LLM-generated vulnerability reports in security triage, statistical machine-text detectors are the wrong instrument: they score provenance, not correctness, and human-written security reports are template-driven and low-perplexity, so they trip the same signal that flags machine text.
Ingested from a paper but not yet reviewed by a human. It is deliberately inert: it does not move any technique’s standing, does not count toward the backtest, and is excluded anywhere a claim would carry weight. Read the source before relying on it.
Capability: Stating false facts confidently
Observed on
Security artifacts with rigid templates and standardized formatting — bug bounty reports, stack traces, patch descriptions; argued, not measured here..
Sources
- Survey. Argued from the low lexical diversity of security report templates plus a cited theoretical result that detector AUC converges to chance as model output distributions approach human writing. No new measurement on security reports is reported here; the causal step from 'formulaic' to 'high false-positive rate' is asserted rather than isolated.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.
Related claims
- Prompt injection defenses that report near-zero attack success on short-context benchmarks lose most of that protection when the injected instruction sits inside a document of thousands to tens of thousands of tokens: fine-tuned separation defenses and detect-localize-remove pipelines still let a large share of injections through on paper review, resume screening, code review and email threads.Following instructions hidden in data · unreviewed
- In synthetic hallucination benchmarks where faithful items are human-written and hallucinated items are LLM-rewrites, a detector's binary score partly measures human-vs-machine style rather than factuality: feeding faithful LLM rewrites of the same truthful articles makes some fine-tuned encoders flag nearly all of them as hallucinated.Whether the measurement made the finding · unreviewed
- For tool-using agents, enforcing injection defense as a deterministic provenance check on sensitive tool parameters — with the only model call reading the trusted user request and never fetched content — makes admission decisions invariant to how an injection is worded, whereas defenses that judge the agent's runtime plan or behavior with a model can be steered by reworded injections.Following instructions hidden in data · unreviewed
- For open-weight agents where attention matrices are accessible, detecting injection by asking which context span actually drove the tool call — aggregating attention over the decision tokens and localizing the guiding span, then checking whether that span's provider has authority for the action — catches unauthorized tool invocations at higher true-positive and lower false-positive rates than static scanners that look for malicious-looking text, because benign-looking injected text is only harmful when it is what the model actually attends to.Following instructions hidden in data · unreviewed
- For computer-use agents operating on a real OS, indirect-injection success rates measured with a fixed hand-written injection template understate vulnerability, because an attacker that adapts phrasing from the agent's own refusal trajectories can reach substantial success on models the fixed template never penetrates.Following instructions hidden in data · unreviewed
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A statistical AI-text detector applied to a mixed corpus of human and LLM bug bounty reports achieves both low false-positive rates on human reports and high separation of unsupported claims from grounded ones. Proposed technique, not catalogued: Deductive coverage scoring of vulnerability claims.