no-thinking-controls-do-not-stop-visible-reasoning CLM-0514observationsingle paperReviewed by AI
Across six models and six prompting interventions on Boolean, multiple-choice and open-ended questions, explicit controls to stop reasoning — a disabled thinking mode or an instruction to answer only — do not reliably eliminate visible inference before the answer: reasoning persists even under strict controls ("thinking inertia"), and it becomes more prevalent as the answer space opens, least on Boolean questions and most on open-ended ones; so a direct answer cannot be assumed from model settings or instructions alone.
Evidence for: Parkinson's law (narrows)
Capability: Cutting to the chase
Observed on
2026, six LLMs; visible pre-answer text only, internal reasoning not measured. Chat assistant.
Sources
- The paper's central finding. Responses are split into a pre-answer trace and a final answer and scored three ways: empty-thinking rate, question-to-preamble relevance, and an LLM-judged explicit-inference rate. The last uses an LLM judge, whose own biases are a known issue.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Evaluations that define "the model did not think" by a disabled thinking mode or by the absence of a long trace measure the wrong thing: a disabled mode may still emit reasoning in the answer, and a long trace may hold filler rather than inference; separating the pre-answer text from the answer and scoring it for emptiness, relevance to the question and explicit inference distinguishes answer-only output, relevant but non-inferential text, and genuine reasoning, which the proxies conflate.Whether the measurement made the finding · reviewed by AI
- On questions whose premises are false or internally contradictory, prompting mid-size instruct models (Qwen2.5-7B, LLaMA-3.1-8B, Gemma3-12B) to reason step by step can lower accuracy below plain answering, because the reasoning chain elaborates from the flawed premise instead of challenging it.Stating false facts confidently · reviewed by AI
- When a model is asked to answer without visible reasoning, rewriting the same question with candidate answers supplied makes answer-only responses easier to produce, and on Boolean and multiple-choice questions answering directly leaves accuracy unchanged — whereas on open-ended questions there is a trade-off, and forcing an answer-only response costs accuracy.Cutting to the chase · reviewed by AI
- In open-weight instruction-tuned models, interventions that suppress caving to user pushback (DPO, SFT on chosen responses, or activation steering) also tend to reduce the model's rate of correcting a wrong answer when genuine supporting evidence is supplied, because the two answer-flip behaviors run on overlapping MLP neurons and attention heads with positively aligned steering directions.Telling the user what they want to hear · reviewed by AI
- For open-source VLMs asked to justify answers on object-removal counterfactual image pairs (A-OKVQA, SNLI-VE), explanations written after the answer is committed (Predict-then-Explain) mention the removed object more in proportion to how much its removal shifts the answer distribution than pre-answer step-by-step CoT does, so pre-answer reasoning is not the more faithful format here.Saying what actually drove the answer · reviewed by AI
Notes
First claim under cutting-to-the-chase. Also filed as evidence on Parkinson's law: offered no reasoning budget, the reasoning did not shrink to fit, which is the low-end case of effort being set by something other than the budget offered.