training-free-mllm-driven-navigation-instruction-following-object-goal-simulated-indoor-environments-gating CLM-0500For training-free MLLM-driven navigation (instruction-following and object-goal) in simulated indoor environments, gating each planner-proposed action on evidential support, geometric reachability and subgoal consistency before dispatch — and refusing a stop request unless semantic, geometric and progress checks all pass — raises success rate over letting the planner act directly, but the stop check buys completion at some cost in path efficiency.
Capability: Turning plans into physical action
Observed on
Zero-shot MLLM planners on VLN-CE R2R/RxR and HM3D-v2/HM3D-OVON; the harness supplies depth, open-vocabulary detection and a deterministic path planner, so results depend on those .
Sources
- Validated by: Cumulative ablation on fixed 100-episode subsets: base agent 46.0 SR (R2R) and 45.0 SR (HM3D-OVON) rose to 64.0 and 55.0 with event memory, graph retrieval and stop validation; stop validation alone added 4.0 and 2.0 SR but lowered OVON SPL from 34.2 to 33.0 and widened its OSR-SR gap from 18 to 19. Full system reported 60.8/53.9/76.0/59.3 SR on R2R/RxR/HM3D-v2/HM3D-OVON versus prior training-free bests.. Measured on four simulated benchmarks plus a qualitative humanoid deployment. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For VLM agents doing multi-phase navigation and pick-and-place in procedurally generated indoor scenes with moving human obstacles, subtask progress runs well ahead of full success — agents reach the first goal and then stall or loop after the phase transition — so success rate alone hides where the failure is.Turning plans into physical action · reviewed by AI
- For long-horizon embodied task planning where a symbolic model defines action preconditions and effects, correcting a violated plan by edits the action model itself implies (inserting the missing navigate or open) raises plan success more per LLM call than returning the violation to the LLM for replanning, and the gain persists for frontier models as well as compact ones.Planning a sequence of actions before acting · reviewed by AI
- For LLM-guided narrow-space parking where an optimal-control solver supplies pass/fail evidence, growing a decision knowledge base offline from accumulated solver and validation failures — retaining an entry only when re-planning then yields a feasible trajectory — raises planning success from 85.1% to 94.7% and closed-loop success from 74.6% to 84.2% over an empty knowledge base, with model weights frozen.Improving its own scaffolding from failures · reviewed by AI
- For routing a video-production request to an ordered chain of registered sub-agents, an 8B open model fine-tuned on descriptor-conditioned routing plans (SFT on verified plans plus GRPO rewarding validity, sub-agent coverage F1 and dependency-consistent order) picks chains more accurately than much larger prompted planners, with the gap widest on long, structurally unfamiliar chains.Getting more from several models than from one · reviewed by AI
- On 102 IPC Blocksworld instances with a 180-second budget, a classical planner (Fast Downward) solved 85%, a model planning directly solved 64%, and the same model choosing one action at a time against a PDDL simulator with observe, reset and retry solved 67%, a consistent but small three-point gain at about 5.7 times the token cost, so interactive simulation helps a little and classical search still wins where a formal model exists.Planning a sequence of actions before acting · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: An ablation on the same benchmarks where removing proposal validation and stop gating leaves success rate unchanged, or where the gating lowers success rate, would show the validation layer is not what produces the gain.