arxiv-2610-11765 SRC-0579 · paperThinking Inertia: LLMs Keep Thinking When Told Not To
Dianqiao Lei, Kevin Qinghong Lin, Pan Lu, Philip Torr, James Zou
Created: 2026-10-08 · Ingested: 2026-10-18
https://arxiv.org/abs/2610.11765(opens in a new tab)Large Language Models (LLMs) increasingly ship with explicit "thinking modes", yet their counterpart, "no-thinking", has received far less attention. We study LLMs' no-thinking behavior along two axes. a. How to measure no-thinking? Prior work typically defines no-thinking through proxies such as a disabled thinking mode or the absence of long traces. These proxies are unreliable: disabled thinking modes may still emit reasoning, while long traces may contain filler rather than genuine inference. We instead normalize each response into a pre-answer trace and final answer, and evaluate it at three levels: (i) Empty-Thinking Rate for strict answer-only compliance; (ii) instruction-aware Question-Pre-answer Relevance for similarity between the question and pre-answer trace; and (iii) LLM-as-judge Explicit Inference Rate for visible explicit inference. Together, these met…
Referenced by
Claims in this catalog that draw on this source, and whether as support or counterpoint.
- supportsEvaluations that define "the model did not think" by a disabled thinking mode or by the absence of a long trace measure the wrong thing: a disabled mode may still emit reasoning in the answer, and a long trace may hold filler rather than inference; separating the pre-answer text from the answer and scoring it for emptiness, relevance to the question and explicit inference distinguishes answer-only output, relevant but non-inferential text, and genuine reasoning, which the proxies conflate.
- supportsAcross six models and six prompting interventions on Boolean, multiple-choice and open-ended questions, explicit controls to stop reasoning — a disabled thinking mode or an instruction to answer only — do not reliably eliminate visible inference before the answer: reasoning persists even under strict controls ("thinking inertia"), and it becomes more prevalent as the answer space opens, least on Boolean questions and most on open-ended ones; so a direct answer cannot be assumed from model settings or instructions alone.
- supportsWhen a model is asked to answer without visible reasoning, rewriting the same question with candidate answers supplied makes answer-only responses easier to produce, and on Boolean and multiple-choice questions answering directly leaves accuracy unchanged — whereas on open-ended questions there is a trade-off, and forcing an answer-only response costs accuracy.