embodied-multi-agent-household-planning-fixed-high-level-skill-interface-team CLM-0563In embodied multi-agent household planning with a fixed high-level skill interface, a team's task success rate overstates coordination quality, because goal predicates can be reached through illegal shortcuts: several of 11 tested MLLMs succeed on more episodes than they produce legal plans on (Claude Haiku 4.5 63.1% success vs 47.2% legal; Qwen3.6-Plus 69.1% vs 56.7%), while others honour constraints yet miss the goal.
Capability: Getting more from several models than from one
Observed on
897 oracle-validated AI2-THOR household instances, 2-4 agents, four coordination constructs (allocation, ordering, mutual exclusion, handoff), centralized image-observation protoco.
Sources
- Validated by: Cross-applying all four scoring functions to 897 GPT-5.6-sol trajectories gave off-diagonal Pearson correlations within [-0.10, 0.16], indicating the four scores are largely non-redundant and not a re-encoding of binary success; per-model success and legality rates diverge in both directions.. Measured on 897 instances across 11 models, plus a 240-instance balanced subset for architecture, observation and team-size ablations. Construct scores are hand-designed formulas calibrated so oracle trajectories score near ceiling; [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Across 41 models and seven multi-agent benchmarks, gains in single-agent capability translate only weakly into multi-agent competence (understanding other agents, joint planning, efficient communication, adaptation); on the multi-agent planning task accuracy rose only about 30% across a family's full scale range, so coordination ability does not emerge from scaling alone and needs its own data and training.Getting more from several models than from one · reviewed by AI
- For long-horizon embodied task planning where a symbolic model defines action preconditions and effects, correcting a violated plan by edits the action model itself implies (inserting the missing navigate or open) raises plan success more per LLM call than returning the violation to the LLM for replanning, and the gain persists for frontier models as well as compact ones.Planning a sequence of actions before acting · reviewed by AI
- In VLM-driven human-aware robot task planning over scene graphs, the bottleneck is inferring and grounding the symbolic goal, not the planning: once both human and robot goals correctly cover the ground truth, a formal multi-agent planner produces a valid, conflict-free joint plan in 99.6% of trials, while end-to-end success with VLM-inferred human goals is 69.7%.Planning a sequence of actions before acting · reviewed by AI
- For multi-step assembly task planning from an action history and a workstation image, prompting a multimodal LLM to emit a structured (JSON) state representation and propagate it through each predicted action — then truncating the plan at the predicted state closest to the inferred goal — raises action executability and lowers final-state error relative to predicting actions alone, at the cost of shorter plans (lower overlap with recorded human action sequences) and roughly eight times the latency.Planning a sequence of actions before acting · reviewed by AI
- In LLM multi-agent systems that reflect after failure, restricting reflection to the single agent responsible for the first incorrect step — located by a trained step-level correctness scorer and paired with a counterfactually generated corrected step — raises task success more than having every agent reflect on the trajectory, because reflections written by agents that behaved correctly enter their memory as misleading guidance for later tasks.Getting more from several models than from one · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A study showing that on these or similar embodied coordination tasks, legal-plan rate and construct scores track task success closely (high correlation, consistent model ranking), so aggregate success loses no diagnostic information.