offering-candidate-answers-makes-answer-only-responses-easier CLM-0515observationsingle paperReviewed by AI
When a model is asked to answer without visible reasoning, rewriting the same question with candidate answers supplied makes answer-only responses easier to produce, and on Boolean and multiple-choice questions answering directly leaves accuracy unchanged — whereas on open-ended questions there is a trade-off, and forcing an answer-only response costs accuracy.
Capability: Cutting to the chase
Observed on
2026, six LLMs; the same questions rewritten across answer spaces. Chat assistant.
Applies when
- Effect on the capability:
- helps, narrowly
- Helps most:
- independent
- Cost:
- low
- Fails when:
- The answer space is open and cannot be enumerated: there, answering directly trades away accuracy, and offering candidates would narrow or give away the answer.
Sources
- The paper's answer-space rewriting experiment and its accuracy results by task type. The "effect" is on answer-only compliance; on accuracy the technique is neutral for closed questions.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Across six models and six prompting interventions on Boolean, multiple-choice and open-ended questions, explicit controls to stop reasoning — a disabled thinking mode or an instruction to answer only — do not reliably eliminate visible inference before the answer: reasoning persists even under strict controls ("thinking inertia"), and it becomes more prevalent as the answer space opens, least on Boolean questions and most on open-ended ones; so a direct answer cannot be assumed from model settings or instructions alone.Cutting to the chase · reviewed by AI
- Evaluations that define "the model did not think" by a disabled thinking mode or by the absence of a long trace measure the wrong thing: a disabled mode may still emit reasoning in the answer, and a long trace may hold filler rather than inference; separating the pre-answer text from the answer and scoring it for emptiness, relevance to the question and explicit inference distinguishes answer-only output, relevant but non-inferential text, and genuine reasoning, which the proxies conflate.Whether the measurement made the finding · reviewed by AI
- On factual QA (817-item TruthfulQA multiple-choice, plus a 300-item Natural Questions short-answer set), a prompt-only three-stage gate in which the model lists the information units a question needs, scores its support for each, and answers only when the mean score clears a threshold lowers the unconditional wrong-answer rate and raises accuracy on answered items for all eleven models tested, but most of the abstentions it produces fall on questions plain chain-of-thought answered correctly (about 69% at the primary operating point), so the gain is bought with lost coverage rather than with selective self-knowledge.Saying it does not know · reviewed by AI