decoder-only-360m-8b-instruct-models-adding-application-supplied-provenance-labels-architectural CLM-0374For decoder-only 360M-8B instruct models, adding application-supplied provenance labels as an architectural signal — origin embeddings plus a learned ring-indexed attention bias that suppresses assistant and user queries attending to untrusted-source keys — cuts indirect-injection attack success on unseen task domains and injection styles further than the same two-stage fine-tuning without the architectural components, even though both are equal in-distribution.
Capability: Following instructions hidden in data
Observed on
Requires the application to be able to tag token spans by source; holds where untrusted external content need not be acted on as instructions, since Ring-3 suppression is unconditi.
Sources
- supportsOrigin Is All You Need: Provenance-Aware Transformers for Structural Trust-Boundary SeparationValidated by: Attack success rate on one in-distribution (SEP, 4,375) and three out-of-distribution injection sets (PI-Attack, DataSentinel, Math-Tutor) across four base models, against an architecture-free same-data SFT baseline and ten published defenses; OOD ASR <=0.05% versus 2.3-17.8% for SFT. Utility via AlpacaEval 2.0 length-controlled win rate against the base model (48.8-60.9% for three models, 42.7% for Mistral). Zeroing the attention bias at inference raises OOD ASR to 12-98%; skipping stage 1 leaves ASR unchanged but collapses win rate.. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- As long as trusted instructions and untrusted retrieved content occupy the same model context, defenses that only change how untrusted text is marked or weighted (instruction hierarchies, spotlighting, content detectors) reduce indirect-injection success but do not remove it; only defenses that isolate contexts or restrict the action set give guarantees, at a cost in the agent's ability to combine information across sources.Following instructions hidden in data · reviewed by AI
- Open-weight instruction-tuned models, told in plain language to not think about a concept while transcribing a sentence, suppress that concept's residual-stream projection enough to evade linear probes, the Jacobian lens, activation oracles and natural-language autoencoders, with no fine-tuning or prompt search — detection at a 1% false-positive budget falls to roughly 5-21% under suppression while engagement is caught near ceiling.Strategic deception and detecting it · reviewed by AI
- For small instruction-tuned models (3-4B) answering closed-form true/false questions under a stated user opinion, scoring each sampled response by how much more common its answer is than the group itself predicted, and using that score as the GRPO reward, reduces answer flips under user pressure and raises accuracy without any labels, roughly matching label-supervised desycophancy fine-tuning at higher compute cost.Telling the user what they want to hear · reviewed by AI
- When fine-tuning 7-8B code models on an LLM-synthesized Verilog instruction corpus, training on a 20-25% subset selected for functional correctness, instruction-solution alignment, and coverage of Verilog structural clusters (AST, CFG, netlist features) yields higher Pass@1 than training on the full corpus, because the automated corpus carries noise and redundancy that filtering removes.Generating and editing working code · reviewed by AI
- For small open-weight backbones (Qwen3-4B) driving long-horizon remote-sensing tool chains, inserting a step-level verifier that checks each intermediate tool output against schema, sub-goal and domain constraints and triggers replanning is the single largest architectural gain, while hierarchical decomposition without domain-aligned tool grouping adds little and can lower final accuracy relative to a flat agent with the same verifier.Completing tasks that take many steps or many hours · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: An adaptive or reworded injection placed in Ring 3 that raises attack success to the architecture-free SFT baseline's level, or a replication where the ring-indexed attention bias adds no OOD margin over the same fine-tuning without it. Drafted stance toward structured-separation-needs-training-and-reduces-rather-than-eliminates: supports -- The architecture-free fine-tuning baseline already cuts in-distribution ASR below 1%, and the added structural separation still needs two-stage fine-tuning to work, so separation's strength comes from training plus structure rather than labels alone.