tool-using-agents-restricting-capabilities-harness-level-after-untrusted-content CLM-0173For tool-using agents, restricting capabilities at the harness level after untrusted content enters the context — revoking or tightening the parameter envelopes of skills that lie on graph paths to deployer-defined forbidden states — cuts indirect-injection attack success further than prompt-level or per-action authorization defenses, and does so with no extra model calls or tokens, because enforcement reads the transition graph rather than the natural-language content.
Evidence for: Postel's law (breaks)
Capability: Following instructions hidden in data
Observed on
Requires a trusted harness that mediates every skill invocation, correct skill transition summaries, and a deployer-written policy naming forbidden states and per-parameter steerab.
Sources
- Measured on four AgentDojo suites with Gemini 2.5 Flash and Llama3.3-70B, against No Defense, Spotlighting, CaMeL and AttriGuard, plus an author-built compositional attack set. Reports 0% attack success on Travel, Banking, Workspace for both backends and 4.8%/14.3% on Slack. Benign utility often falls relative to no defense; fractional-flow restriction retains more capability than binary at the same attack success. Authors' own system and own compositional benchmark; [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For tool-using agents, a safety harness searched separately for each model-and-domain deployment (jointly evolving a natural-language policy and executable pre/post-tool-call code, screened by a fresh-context adversarial reviewer) reaches a better safety-utility trade-off than fixed expert-designed defenses ported unchanged across deployments, because domain semantics determine which safety relations and trajectory state must be checked while the victim model's failure behavior determines whether a check should be semantic or deterministic and where in execution it must run.Following instructions hidden in data · reviewed by AI
- For tool-using agents, enforcing injection defense as a deterministic provenance check on sensitive tool parameters — with the only model call reading the trusted user request and never fetched content — makes admission decisions invariant to how an injection is worded, whereas defenses that judge the agent's runtime plan or behavior with a model can be steered by reworded injections.Following instructions hidden in data · reviewed by AI
- As long as trusted instructions and untrusted retrieved content occupy the same model context, defenses that only change how untrusted text is marked or weighted (instruction hierarchies, spotlighting, content detectors) reduce indirect-injection success but do not remove it; only defenses that isolate contexts or restrict the action set give guarantees, at a cost in the agent's ability to combine information across sources.Following instructions hidden in data · reviewed by AI
- For open-weight agents where attention matrices are accessible, detecting injection by asking which context span actually drove the tool call — aggregating attention over the decision tokens and localizing the guiding span, then checking whether that span's provider has authority for the action — catches unauthorized tool invocations at higher true-positive and lower false-positive rates than static scanners that look for malicious-looking text, because benign-looking injected text is only harmful when it is what the model actually attends to.Following instructions hidden in data · reviewed by AI
- For tool-integrated LLM agents where the attack works by adding attacker-controlled tools, overwriting the agent's visible toolset with the saved user toolset immediately before planning drives attack success to near zero across direct and indirect injection, memory poisoning and backdoor attacks, but its effect on benign task success is model-dependent — raising it for 7-9B open models and lowering it for GPT-3.5/GPT-4.Following instructions hidden in data · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: An evaluation on comparable agent benchmarks where post-contamination capability restriction leaves attack success no lower than CaMeL/AttriGuard-style per-action authorization, or where the utility cost of confinement is so large that the defended agent completes fewer tasks than a prompt-level defense at equal attack success. Proposed technique, not catalogued: reachability-based capability confinement after contamination.