tool-using-agents-long-trajectory-benchmarks-injection-defense-audits-each-candidate CLM-0226For tool-using agents on long-trajectory benchmarks, an injection defense that audits each candidate action against a one-step-ahead predicted tool set holds attack success low while keeping task utility near the attack-free level, whereas defenses that fix a global plan or filter whole tool outputs lose utility because the predicted full trajectory almost never matches the executed one once environmental feedback arrives.
Capability: Following instructions hidden in data
Observed on
Multi-step tool agents with untrusted observations; measured on AgentDyn and AgentDojo with GPT-4o-mini, Gemini-2.5-Flash and DeepSeek-V4-Flash backends; requires log-probability a.
Sources
- Measured against 9 defense baselines on two benchmarks with three backends, reporting attack-free utility, under-attack utility and attack success rate. Planning-reference coverage is reported directly: step-wise next-tool prediction 58.7-72.2% no-attack and 54.2-69.6% under attack, versus global plans 9.7-16.1% no-attack and 0.00% under attack. Ablations show removing argument-level localization causes the largest degradation (ASR 3.0 to 24.5 on DailyLife). All numbers from the paper's own implementation; [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For long-horizon tool-calling agents (tasks needing five or more calls), embedding state-transition cues — preconditions, invariants, completion states — into the descriptions of the tools on the intended chain is what actually steers the agent's trajectory; runtime corrective text appended to tool results only patches residual drift, and the plausible user prompt alone (persona, deadlines, format constraints) does not establish the trajectory at all.Following instructions hidden in data · reviewed by AI
- For tool-using agents, enforcing injection defense as a deterministic provenance check on sensitive tool parameters — with the only model call reading the trusted user request and never fetched content — makes admission decisions invariant to how an injection is worded, whereas defenses that judge the agent's runtime plan or behavior with a model can be steered by reworded injections.Following instructions hidden in data · reviewed by AI
- For long-horizon LLM agents on tool-use and terminal benchmarks, token-level confidence (predictive entropy, sequence probability) barely separates successful from failed trajectories, and the separable signal lives in the dependency structure of the trajectory: scoring a trajectory by propagating per-step uncertainty over a graph of reasoning, tool-call and environment-feedback nodes raises failure-detection AUROC over both local-confidence and sequence-based agent UQ baselines.Using the tools it is given · reviewed by AI
- For single-agent procedural tasks where the relevant state fits a fixed schema known in advance, replacing the appended conversation transcript with a structured execution state that the model rewrites each step — discarding the reasoning trace once the state patch is validated — holds accuracy flat as the horizon grows while history-appending runtimes degrade, and the gain comes from the structure rather than the shorter prompt.Completing tasks that take many steps or many hours · reviewed by AI
- For tool-using agents, a safety harness searched separately for each model-and-domain deployment (jointly evolving a natural-language policy and executable pre/post-tool-call code, screened by a fresh-context adversarial reviewer) reaches a better safety-utility trade-off than fixed expert-designed defenses ported unchanged across deployments, because domain semantics determine which safety relations and trajectory state must be checked while the victim model's failure behavior determines whether a check should be semantic or deterministic and where in execution it must run.Following instructions hidden in data · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: Show a global-plan or whole-output-filtering defense that matches this method's under-attack utility at equal attack success on long-trajectory agent benchmarks, or show that step-wise next-tool prediction coverage is no higher than global plan coverage on executed trajectories. Proposed technique, not catalogued: pre-execution action auditing against a step-wise tool prior. Automatic check flagged: figures not in the source: 54.2, 9.7.