llm-assisted-soc-pipeline-where-injections-planted-log-fields-deterministic CLM-0418In an LLM-assisted SOC pipeline where injections are planted in log fields, a deterministic SIEM pre-filter (regex decoders plus length truncation) catches only the structurally obvious payloads while an LLM self-check input/output rail catches most semantic ones, so the two layers block disjoint attack classes and neither alone is sufficient.
Capability: Following instructions hidden in data
Observed on
Prototype AI-SOC with Wazuh decoders plus NeMo Guardrails, local Gemma 3 12B analyst model, 20 hand-crafted injected alerts across four MITRE ATLAS vectors..
Sources
- Validated by: On 20 injected Wazuh alerts, Layer 1 intercepted 3 and Layer 2 intercepted 18 of 20; two evasions remained (one tool-abuse, one instruction-override jailbreak). No baseline of undefended pipeline success rate reported, no latency numbers.. Measured on a single hand-built dataset of 20 injections against one 12B model, with per-layer counts but no undefended baseline, no adaptive attacker, and no latency or false-positive measurement; layer-count split is the only quantitative evidence.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For LLM data-preparation pipelines in specialist domains (math, finance, law, medicine), diagnosing a small pilot set of generated samples against domain-selected quality criteria and rewriting the individual pipeline components (chunking, instruction builder, response generator) before full-scale generation yields training data with higher downstream fine-tuning utility than post-hoc filtering or one-shot agent/skill generators using the same backbone — but the gain is non-monotonic in rounds, peaking at two and degrading after.Improving its own scaffolding from failures · reviewed by AI
- A linear probe on a tool-calling model's hidden state at the last generated token detects incorrect tool-calls — including wrong-but-well-typed argument values that parsers and logs do not catch — and works better on larger models and at middle-to-late layers than at the final layer.Using the tools it is given · reviewed by AI
- For decoder-only 360M-8B instruct models, adding application-supplied provenance labels as an architectural signal — origin embeddings plus a learned ring-indexed attention bias that suppresses assistant and user queries attending to untrusted-source keys — cuts indirect-injection attack success on unseen task domains and injection styles further than the same two-stage fine-tuning without the architectural components, even though both are equal in-distribution.Following instructions hidden in data · reviewed by AI
- For tool-using agents, enforcing injection defense as a deterministic provenance check on sensitive tool parameters — with the only model call reading the trusted user request and never fetched content — makes admission decisions invariant to how an injection is worded, whereas defenses that judge the agent's runtime plan or behavior with a model can be steered by reworded injections.Following instructions hidden in data · reviewed by AI
- Prompt injection defenses that report near-zero attack success on short-context benchmarks lose most of that protection when the injected instruction sits inside a document of thousands to tens of thousands of tokens: fine-tuned separation defenses and detect-localize-remove pipelines still let a large share of injections through on paper review, resume screening, code review and email threads.Following instructions hidden in data · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: A study showing the deterministic pre-filter catches the same injections the semantic rails catch (or vice versa), so removing one layer leaves attack success unchanged on a larger and adversarially adaptive injection set.