small-open-weight-backbones-qwen3-4b-driving-long-horizon-remote-sensing-tool-chains CLM-0489observationsingle paperReviewed by AI
For small open-weight backbones (Qwen3-4B) driving long-horizon remote-sensing tool chains, inserting a step-level verifier that checks each intermediate tool output against schema, sub-goal and domain constraints and triggers replanning is the single largest architectural gain, while hierarchical decomposition without domain-aligned tool grouping adds little and can lower final accuracy relative to a flat agent with the same verifier.
Capability: Completing tasks that take many steps or many hours
Observed on
Two remote-sensing agent benchmarks (Earth-Bench, ThinkGeo), Qwen3-4B/8B shared-parameter backbone with role prompts, fixed tool registry; all components also trained with domain S.
Sources
- supportsHiRS-Agent: A Hierarchical Multi-Agent System for Reliable Long-Horizon Remote Sensing Task SolvingValidated by: Architecture ablation on Earth-Bench with Qwen3-4B: adding the verifier to a flat agent raised Tool-Exact-Match from 6.84/19.42 to 29.05/30.14 and Accuracy from 16.53/20.16 to 35.89/31.45; removing the verifier from the full system cut Exact by 11.30/12.43 and Accuracy by 4.03/2.82.. Measured, single paper, one ablation table on one backbone (Qwen3-4B) and one benchmark; the verifier is entangled with the training stages applied to the same checkpoint, and the counter-intuitive generic-hierarchy result rests on a single row with no variance reported.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- On a suite of verifier-checked tool-using ReAct task families (arithmetic, SQL analytics, security judgment, URL grounding, dependency planning, orchestration, tool policy), a single task-weighted aggregate accuracy hides inverted per-family rankings: the model that ranks last overall can lead both URL-grounding families while scoring lowest on structured SQL and multi-step tool families, which spread the four tested models by more than sixty points.Whether the measurement made the finding · reviewed by AI
- For LLM agents that must compose and then execute domain tool workflows, training separate adapters for the two roles splits the gains cleanly: supervised fine-tuning of the planner on reference tool sequences mainly improves workflow-order fidelity, while node-level RL on argument-match rewards mainly improves tool-argument grounding and final-answer quality, and combining both beats either on a 9B backbone.Planning a sequence of actions before acting · reviewed by AI
- When a small (1.7B) LLM is fine-tuned to act as an ensemble router that assigns per-sample weights across candidate time-series forecasters, chain-of-thought supervision built by rule-templating known oracle weights against extracted temporal features matches or beats CoT generated by a frontier API model, and removing the CoT step costs far more on unseen candidate model pools than on in-distribution ones.Predicting future events · reviewed by AI
- On 2,154 expert-curated multiple-choice questions across eight architectural categories, 11 current models all score above 90% zero-shot (best about 94%), with quality-attribute and pattern questions near ceiling, but accuracy varies by category and drops for smaller models on the harder tiers (to about 85-87%), so declarative architectural knowledge is largely present while the benchmark cannot show whether it is applied in design.Making design decisions that hold up as a system grows · reviewed by AI
- For long-horizon LLM agents on tool-use and terminal benchmarks, token-level confidence (predictive entropy, sequence probability) barely separates successful from failed trajectories, and the separable signal lives in the dependency structure of the trajectory: scoring a trajectory by propagating per-step uncertainty over a graph of reasoning, tool-call and environment-feedback nodes raises failure-detection AUROC over both local-confidence and sequence-based agent UQ baselines.Using the tools it is given · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: An ablation on the same benchmarks where removing step-level verification leaves tool-exactness and final accuracy roughly unchanged, or where hierarchy without domain grouping beats a flat verified agent.