vlm-agents-doing-multi-phase-navigation-pick-and-place-procedurally-generated-indoor CLM-0655observationsingle paperReviewed by AI
For VLM agents doing multi-phase navigation and pick-and-place in procedurally generated indoor scenes with moving human obstacles, subtask progress runs well ahead of full success — agents reach the first goal and then stall or loop after the phase transition — so success rate alone hides where the failure is.
Capability: Turning plans into physical action
Observed on
Two VLM agents (Qwen3-VL-30B-A3B-Thinking, Gemini 3.1 Pro) on 333 tasks across three tiers varying subtask count and initial target visibility, 150-action/50-call budget, kinematic.
Sources
- Validated by: Not measured as an intervention; the paper reports that SP exceeds SR on the two-phase tiers (Qwen 35.5% SP vs 18.2% SR on L2; 15.0% vs 4.5% on L3; Gemini 36.4% vs 23.6% and 27.7% vs 18.2%) and shows a trajectory where the agent reaches goal 1 then ends 8.0 m from goal 2.. Measured on 333 auto-generated tasks in 113 Infinigen Indoors scenes, two models only, differing in architecture, training and serving stack, so the cross-model comparison is diagnostic not controlled; the progress-exceeds-success gap holds within each model on both two-phase tiers. [truncated]
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For training-free MLLM-driven navigation (instruction-following and object-goal) in simulated indoor environments, gating each planner-proposed action on evidential support, geometric reachability and subgoal consistency before dispatch — and refusing a stop request unless semantic, geometric and progress checks all pass — raises success rate over letting the planner act directly, but the stop check buys completion at some cost in path efficiency.Turning plans into physical action · reviewed by AI
- For deciding when a robot skill has finished and the next subtask should begin, general-purpose VLMs prompted zero-shot are near useless on long-horizon mobile manipulation (under 4% of transitions detected on BEHAVIOR-1K, including a frontier proprietary model), while a sub-1B student fine-tuned on compact self-explanations condensed from teacher reasoning over annotated demonstrations detects about 90% and lifts closed-loop task success from 0.30 to 0.53 against a 0.63 oracle.Turning plans into physical action · reviewed by AI
- In VLM-driven human-aware robot task planning over scene graphs, the bottleneck is inferring and grounding the symbolic goal, not the planning: once both human and robot goals correctly cover the ground truth, a formal multi-agent planner produces a valid, conflict-free joint plan in 99.6% of trials, while end-to-end success with VLM-inferred human goals is 69.7%.Planning a sequence of actions before acting · reviewed by AI
- In multi-agent LLM workflows with a mixed-size model zoo, choosing which model instantiates each agent step-by-step from an online estimate of task-progress gain plus remaining time and cost budget keeps task quality near the best single-model policy at lower energy than one-shot query-level routing or stall-triggered escalation.Getting more from several models than from one · reviewed by AI
- In embodied multi-agent household planning with a fixed high-level skill interface, a team's task success rate overstates coordination quality, because goal predicates can be reached through illegal shortcuts: several of 11 tested MLLMs succeed on more episodes than they produce legal plans on (Claude Haiku 4.5 63.1% success vs 47.2% legal; Qwen3.6-Plus 69.1% vs 56.7%), while others honour constraints yet miss the goal.Getting more from several models than from one · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A benchmark of the same kind where subtask progress and success rate track each other closely for several models, i.e. agents that reach the first goal almost always complete the second.