arxiv-2602-03900 SRC-0345 · paper

Knowledge Model Prompting Increases LLM Performance on Planning Tasks

Created: 2026-02-03 · Ingested: 2026-09-19

Large Language Models (LLM) can struggle with reasoning ability and planning tasks. Many prompting techniques have been developed to assist with LLM reasoning, notably Chain-of-Thought (CoT); however, these techniques, too, have come under scrutiny as LLMs' ability to reason at all has come into question. Borrowing from the domain of cognitive and educational science, this paper investigates whether the Task-Method-Knowledge (TMK) framework can improve LLM reasoning capabilities beyond its previously demonstrated success in educational applications. The TMK framework's unique ability to captur

In brief

Replacing the natural-language domain description in a planning prompt with a JSON-serialized Task-Method-Knowledge (TMK) structure lifts plan accuracy most where linguistic cues are absent, and reverses the usual ordering in which models do better on semantically misleading tasks than on opaque symbolic ones.

The test bed is PlanBench Blocksworld in 3 variants: Classic (canonical action names), Mystery (unrelated words), and Random (opaque alphanumeric identifiers), scored as fully correct stepwise plans by classical plan validators. Models are GPT4, GPT4o, o1mini, o1, and GPT5; TMK prompts are one-shot with a non-matching example, compared against the best of zero- and one-shot plain-text numbers from the public PlanBench leaderboard.

o1 on Random rises from 31.5 to 97.33, a 65.8% gain, exceeding its own Mystery score under TMK (83.3 versus 74.3 plain text). Gains are smaller elsewhere: GPT4o Classic 35.5 to 45.3, GPT5 Random 92.5 to 99.0. o1mini regresses on Mystery, 19.1 to 16.83.

The effect is measured, but the baseline comes from a leaderboard at a different shot count rather than a matched rerun, the TMK prompts differ slightly per variant, and the code-pathway explanation is argued, not tested.

Only OpenAI models and only Blocksworld; Logistics, multi-agent planning, and comparisons to HTN or BDI prompts are untested.

If you prompt planners, the serialization format may matter more than the example.

Written from the abstract by claude-opus-5 on 2026-10-17, with every figure checked against it. Not a substitute for the paper.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsPrompting with a Task-Method-Knowledge model, which spells out what to do, how, and why in a hierarchical structure, lifted o1 on the obfuscated Random Blocksworld set of PlanBench from about 32% to 97%, and o1-mini from about 9% to 27%, gains far beyond the roughly 1% that zero-shot chain-of-thought gave on PlanBench in earlier work; the smaller model did not benefit on the Mystery variant.