llm-driven-robot-manipulation-where-every-tool-call-schema-validated-every CLM-0591observationsingle paperReviewed by AI
For LLM-driven robot manipulation where every tool call is schema-validated and every motion kinematically and collision-checked before execution, execution-time recovery mechanisms (retrying a failed plan with an alternative candidate, resetting to a collision-checked safe pose between stages) contribute more to end-to-end task success than canonical action sequencing, because most residual failures are transient planning or IK failures rather than wrong orderings.
Capability: Turning plans into physical action
Observed on
Single real robot (UFactory 850) with eye-to-hand RGB-D perception, four pouring tasks plus one grasp-and-place task, 10 physical trials per condition..
Sources
- supportsBridging Semantics and Physics with Constrained LLMs for Safe and Trustworthy Robotic ManipulationValidated by: End-to-end success over 10 physical trials per task: 72.5% mean vs 50.0% for a scripted FSM/BT policy and 47.5% for the same LLM without canonical sequencing; ablations removing safe-pose reset (-32.5 points) and retry (-47.5 points) versus removing the task graph (-25.0 points).. Measured, but on only 10 trials per condition on one robot, so each trial is 10 percentage points and per-task differences are noisy; the authors say so. [truncated]
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For open-source LLM agents driving a ROS robot, having the model emit a pseudo-code tool-call plan up front, storing it in a persistent scratchpad, and then restricting each tool invocation to the plan's next step raises task success most on the longest multi-step tasks and for the smaller open models (GPT-OSS-20B on the hardest task rose from 35% with a persistent plan alone to 55% with the constraint), while a strong proprietary model gains little.Completing tasks that take many steps or many hours · reviewed by AI
- For long-horizon embodied task planning where a symbolic model defines action preconditions and effects, correcting a violated plan by edits the action model itself implies (inserting the missing navigate or open) raises plan success more per LLM call than returning the violation to the LLM for replanning, and the gain persists for frontier models as well as compact ones.Planning a sequence of actions before acting · reviewed by AI
- When a multimodal LLM acts as the decision core of a simulated humanoid reacting to sudden household hazards, picking the correct reaction primitive does not mean the reaction is executable: even on correctly classified scenes, hands land a median 0.48 m from the true impact point, 89% of misses fall short, and in 49% of those decisions the body never steps forward and stands beyond arm's reach.Turning plans into physical action · reviewed by AI
- For multi-step assembly task planning from an action history and a workstation image, prompting a multimodal LLM to emit a structured (JSON) state representation and propagate it through each predicted action — then truncating the plan at the predicted state closest to the inferred goal — raises action executability and lowers final-state error relative to predicting actions alone, at the cost of shorter plans (lower overlap with recorded human action sequences) and roughly eight times the latency.Planning a sequence of actions before acting · reviewed by AI
- For long-horizon LLM agents on tool-use and terminal benchmarks, token-level confidence (predictive entropy, sequence probability) barely separates successful from failed trajectories, and the separable signal lives in the dependency structure of the trajectory: scoring a trajectory by propagating per-step uncertainty over a graph of reasoning, tool-call and environment-feedback nodes raises failure-detection AUROC over both local-confidence and sequence-based agent UQ baselines.Using the tools it is given · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: A replication with a larger trial count in which removing canonical sequencing costs more end-to-end success than removing retry and safe-pose reset, or in which retry gives no measurable gain once verification is in place.