filtering-unsupported-llm-generated-vulnerability-reports-security-triage-statistical-machine-text CLM-0047For filtering unsupported LLM-generated vulnerability reports in security triage, statistical machine-text detectors are the wrong instrument: they score provenance, not correctness, and human-written security reports are template-driven and low-perplexity, so they trip the same signal that flags machine text.
Capability: Stating false facts confidently
Observed on
Security artifacts with rigid templates and standardized formatting — bug bounty reports, stack traces, patch descriptions; argued, not measured here..
Sources
- Survey. Argued from the low lexical diversity of security report templates plus a cited theoretical result that detector AUC converges to chance as model output distributions approach human writing. No new measurement on security reports is reported here; the causal step from 'formulaic' to 'high false-positive rate' is asserted rather than isolated.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Prompt injection defenses that report near-zero attack success on short-context benchmarks lose most of that protection when the injected instruction sits inside a document of thousands to tens of thousands of tokens: fine-tuned separation defenses and detect-localize-remove pipelines still let a large share of injections through on paper review, resume screening, code review and email threads.Following instructions hidden in data · reviewed by AI
- As long as trusted instructions and untrusted retrieved content occupy the same model context, defenses that only change how untrusted text is marked or weighted (instruction hierarchies, spotlighting, content detectors) reduce indirect-injection success but do not remove it; only defenses that isolate contexts or restrict the action set give guarantees, at a cost in the agent's ability to combine information across sources.Following instructions hidden in data · reviewed by AI
- In synthetic hallucination benchmarks where faithful items are human-written and hallucinated items are LLM-rewrites, a detector's binary score partly measures human-vs-machine style rather than factuality: feeding faithful LLM rewrites of the same truthful articles makes some fine-tuned encoders flag nearly all of them as hallucinated.Whether the measurement made the finding · reviewed by AI
- When a benign GitHub issue is augmented with a plausible-sounding maintenance requirement that specifies a helper reading an environment variable into a dangerous sink, repository-level APR agents produce patches that fix the original bug and pass the project test suite while containing the requested vulnerability in roughly half of tasks, and the strongest instruction-following coding models are more susceptible, not less.Writing secure code and dependencies · reviewed by AI
- When a model is asked to audit many documents for planted errors in a single large-batch prompt, recall of known contaminants collapses and the model reports confident fabricated findings — invented absurd contaminants that appear in no document — rather than signalling incomplete processing.Stating false facts confidently · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A statistical AI-text detector applied to a mixed corpus of human and LLM bug bounty reports achieves both low false-positive rates on human reports and high separation of unsupported claims from grounded ones. Proposed technique, not catalogued: Deductive coverage scoring of vulnerability claims.