arxiv-2608-28476 · paper

ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL

Created: 2026-08-28 · Ingested: 2026-09-09

Long-horizon agentic tasks require large language models (LLMs) to iteratively retrieve, integrate, and maintain dispersed information across multi-turn interactions, but preserving all interaction histories leads to a continuously growing working context. Recent proactive context management methods allow models to edit their own working context with specialized tools, yet they still face three key limitations: (1) a limited toolset restricted to search, deletion, and summarization, with no support for global planning, long-term memory, and adaptive compression; (2) inefficient exploration tha

In brief

Letting an agent edit its own working context works better when credit is assigned to individual edit actions rather than to whole trajectories, and when the toolset includes planning, long-term memory, and soft offloading rather than just search, delete, and summarize. ContextPilot adds tools like memorize/readMemory, foldHistory, compressContext, and plan, then trains with context-aware partial rollout (branching at edits with high context-length and entropy variation) plus snapshot-level advantages under GRPO.

Evaluation covers long-context QA (NovelQA, ∞Bench, LongMemEval-S, BrowseComp+) with Qwen3-8B, Qwen3-14B, Gemma4-E4B-it, and deep search (BrowseComp, BrowseComp-ZH, GAIA, xBench-DS) with WebSailor-7B and WebExplorer-8B, all in a 32K window.

ContextPilot-8B-RL beats StateLM-8B-RL by 3.55 average points; on deep search it beats SUPO by 1.51 average points. RL adds 3.62 points over SFT on average, and 5.34 on BrowseComp+. Input length per turn stabilizes near 8K–10K tokens where WebExplorer-8B grows to about 30K.

Effects are measured, with 3 runs per method, cumulative tool ablations (63.49% to 80.96% on BrowseComp+), and an RL-component ablation showing entropy-only branching hurts BrowseComp+ by 1.32 points. Tool ablations use only Qwen3.5-397B-A17B, and some baselines are quoted from prior work rather than rerun.

Untested outside QA and search: no coding or GUI agents, and hyperparameters were not swept.

Written from the abstract by claude-opus-5 on 2026-09-09, with every figure checked against it. Not a substitute for the paper.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsHanding an agent context-editing tools (delete, summarize, note, fold) by prompt alone makes mid-size models worse than leaving their context untouched — Qwen3-8B, Qwen3-14B and Gemma4-E4B all scored lower on long-context QA with the tools than without them; only after fine-tuning on trajectories that use the tools does the toolset become a gain.