asked-generate-full-procedural-execution-graph-unseen-task-recipe CLM-0548When asked to generate a full procedural execution graph for an unseen task (recipe) in a specific creator's style, few-shot LLMs assign ingredients to steps better than a statistical two-stage pipeline but predict the number of steps worse, and swapping in a larger LLM backbone does not close that step-count gap.
Capability: Planning a sequence of actions before acting
Observed on
Culinary procedural graphs extracted from video transcripts (97 creators, >5,000 videos), evaluated by Ingredient F1 and normalized step-count error; GPT-5.4, GPT-4.1-mini, Qwen3.6.
Model cards: GPT-4
Sources
- Validated by: Compared against zero-shot LLM, few-shot LLM, autoregressive step-by-step, and N-gram baselines on Ingredient F1, normalized edge F1 and step error over long- and short-form splits; the two-stage model had the lowest step error (0.350 long, 0.332 short vs 0.373/0.351 few-shot) and lower IF1 (0.294/0.348 vs 0.344/0.385). A persona-swap control (other creator's priors, same dish) cost 0.06 nEF1 (p<1e-54).. Measured on one dataset in one domain, single author, with a persona-swap control that rules out generic culinary structure for the two-stage result. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For LLM data-preparation pipelines in specialist domains (math, finance, law, medicine), diagnosing a small pilot set of generated samples against domain-selected quality criteria and rewriting the individual pipeline components (chunking, instruction builder, response generator) before full-scale generation yields training data with higher downstream fine-tuning utility than post-hoc filtering or one-shot agent/skill generators using the same backbone — but the gain is non-monotonic in rounds, peaking at two and degrading after.Improving its own scaffolding from failures · reviewed by AI
- In multi-agent LLM workflows with a mixed-size model zoo, choosing which model instantiates each agent step-by-step from an online estimate of task-progress gain plus remaining time and cost budget keeps task quality near the best single-model policy at lower energy than one-shot query-level routing or stall-triggered escalation.Getting more from several models than from one · reviewed by AI
- When an agent builds a whole Python repository from natural-language requirements alone, refining component boundaries with explicit cohesion and coupling thresholds until no split or merge is triggered produces higher functionality coverage and test pass rate than letting the LLM decide restructuring actions for a fixed budget of rounds, because unconstrained LLM-decided actions keep over-decomposing past the point of a coherent modular structure.Generating and editing working code · reviewed by AI
- In multi-agent LLM pipelines over reasoning and code tasks with a pool of 3B-32B open-weight backbones, conditioning each per-step routing decision on a learned filtered memory rather than the full accumulated trajectory leaves accuracy about the same while cutting inference cost by roughly 40-50%, so most of the accumulated history is cost without accuracy value.Getting more from several models than from one · reviewed by AI
- For LLM agents that must compose and then execute domain tool workflows, training separate adapters for the two roles splits the gains cleanly: supervised fine-tuning of the planner on reference tool sequences mainly improves workflow-order fidelity, while node-level RL on argument-match rewards mainly improves tool-argument grounding and final-answer quality, and combining both beats either on a 9B backbone.Planning a sequence of actions before acting · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: A few-shot LLM (or a larger backbone) that matches or beats the structured two-stage pipeline on step-count error while keeping its ingredient-assignment advantage on the same graphs.