handing-agent-context-editing-tools-delete-summarize-note-fold-prompt CLM-0067Handing an agent context-editing tools (delete, summarize, note, fold) by prompt alone makes mid-size models worse than leaving their context untouched — Qwen3-8B, Qwen3-14B and Gemma4-E4B all scored lower on long-context QA with the tools than without them; only after fine-tuning on trajectories that use the tools does the toolset become a gain.
Evidence for: Braess's paradox (holds), Lehman's law of increasing complexity (narrows)
Capability: Keeping its own context clean
Observed on
Long-context QA benchmarks (NovelQA, InfinityBench En.MC, LongMemEval-S, BrowseComp+) with 8B-14B open models capped at a 32K working window; a very large model (Qwen3.5-397B-A17B).
Sources
- Measured across three base models and four benchmarks, three runs each, against a no-tools baseline at the model's full 128K window. Average scores dropped for every base model with prompt-only tools; SFT and RL versions of the same models exceeded the no-tools baseline. Does not isolate whether the loss comes from wrong tool arguments or from bad editing decisions, though a separate analysis shows high early invocation-failure rates for memory and offloading tools.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For long-horizon tool-calling agents (tasks needing five or more calls), embedding state-transition cues — preconditions, invariants, completion states — into the descriptions of the tools on the intended chain is what actually steers the agent's trajectory; runtime corrective text appended to tool results only patches residual drift, and the plausible user prompt alone (persona, deadlines, format constraints) does not establish the trajectory at all.Following instructions hidden in data · reviewed by AI
- When a tool-using agent is trained with outcome-only RL and a surface prompt cue co-occurs with one tool in most training examples, the agent invokes that tool on tasks where it is useless, but only for tools it has already learned to use well and mainly when the cue is semantically related to the tool.Using the tools it is given · reviewed by AI
- For LLM agents that must compose and then execute domain tool workflows, training separate adapters for the two roles splits the gains cleanly: supervised fine-tuning of the planner on reference tool sequences mainly improves workflow-order fidelity, while node-level RL on argument-match rewards mainly improves tool-argument grounding and final-answer quality, and combining both beats either on a 9B backbone.Planning a sequence of actions before acting · reviewed by AI
- For tool-integrated LLM agents where the attack works by adding attacker-controlled tools, overwriting the agent's visible toolset with the saved user toolset immediately before planning drives attack success to near zero across direct and indirect injection, memory poisoning and backdoor attacks, but its effect on benign task success is model-dependent — raising it for 7-9B open models and lowering it for GPT-3.5/GPT-4.Following instructions hidden in data · reviewed by AI
- On rubric-graded long-context tasks, most of the gain from a context- compilation harness comes from extracting the context's rules, exact terms and output spec into an explicit checklist placed in the prompt, not from the executable verifiers built on top of it — and the gain appears only on rule-dense tasks and on models with enough capacity, disappearing or reversing on open-ended tasks and on a small-activation model.Losing information in long inputs · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A prompt-only agent given the same context-editing toolset matching or beating the same base model's no-tool score on these long-context QA benchmarks at 8B-14B scale. Proposed technique, not catalogued: RL-trained proactive context management with action-level credit assignment.