training-models-rank-system-user-tool-instructions-bymechanismsingle paper
Training models to rank system, user, and tool instructions by privilege improves robustness to injections in tool outputs.
Capability: Following instructions hidden in data
Sources
- Training models to rank system, user, and tool instructions by privilege improves robustness to injections in tool outputs.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.
Related claims
- For tool-using agents, enforcing injection defense as a deterministic provenance check on sensitive tool parameters — with the only model call reading the trusted user request and never fetched content — makes admission decisions invariant to how an injection is worded, whereas defenses that judge the agent's runtime plan or behavior with a model can be steered by reworded injections.Following instructions hidden in data · unreviewed
- For tool-using agents, restricting capabilities at the harness level after untrusted content enters the context — revoking or tightening the parameter envelopes of skills that lie on graph paths to deployer-defined forbidden states — cuts indirect-injection attack success further than prompt-level or per-action authorization defenses, and does so with no extra model calls or tokens, because enforcement reads the transition graph rather than the natural-language content.Following instructions hidden in data · unreviewed
- Self-supervised training on tool-call examples teaches a model when and how to call simple tools, narrowing the gap on tasks that need them.Using the tools it is given
- A linear probe on a tool-calling model's hidden state at the last generated token detects incorrect tool-calls — including wrong-but-well-typed argument values that parsers and logs do not catch — and works better on larger models and at middle-to-late layers than at the final layer.Using the tools it is given · unreviewed
- For computer-use agents operating on a real OS, indirect-injection success rates measured with a fixed hand-written injection template understate vulnerability, because an attacker that adapts phrasing from the agent's own refusal trajectories can reach substantial success on models the fixed template never penetrates.Following instructions hidden in data · unreviewed