llm-gives-answer-plus-explanation-deleting-input-those-concepts CLM-0240mechanismsingle paperReviewed by AI
When an LLM gives an answer plus an explanation, deleting from the input those concepts the explanation does not credit and re-querying on the reduced input raises measured faithfulness over both standard prompting and prompts that ask for faithful explanations, because the re-query removes the unmentioned factors that could have influenced the original answer.
Capability: Saying what actually drove the answer
Observed on
Tested on two datasets with inputs decomposable into discrete concepts, across multiple model families, with two faithfulness metrics; targets incompleteness (omitted influences), .
Sources
- Validated by: Two faithfulness metrics on two datasets across multiple model families, compared against standard prompting and faithfulness-encouraging prompting.. Measured, but the abstract and introduction alone give no numbers, no dataset names, and no model list; the design does not address unsoundness and assumes concepts can be cleanly segmented and deleted without changing the question.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- When an LLM answers a question and gives an explanation, removing from the input the concepts its explanation did not credit and re-querying on the reduced input raises measured explanation faithfulness over both standard prompting and prompts that ask for faithfulness, because the removal deletes the unmentioned influences rather than asking the model to report them.Saying what actually drove the answer · reviewed by AI
- Unlearning methods that make a model answer "I don't know" about a forget set can leave the knowledge in place: the strongest IDK-style fine-tuning method still answered those same questions accurately when probed in a different format, and across nine unlearning methods on three model families none met a joint standard of utility and honesty on retained knowledge plus consistent, acknowledged forgetting; aligning representations after feature randomization (ReVa) roughly doubled the rejection rate that survives a second conversational turn.Retaining skills and safety after fine-tuning · reviewed by AI
- For open-source VLMs asked to justify answers on object-removal counterfactual image pairs (A-OKVQA, SNLI-VE), explanations written after the answer is committed (Predict-then-Explain) mention the removed object more in proportion to how much its removal shifts the answer distribution than pre-answer step-by-step CoT does, so pre-answer reasoning is not the more faithful format here.Saying what actually drove the answer · reviewed by AI
- On factual QA (817-item TruthfulQA multiple-choice, plus a 300-item Natural Questions short-answer set), a prompt-only three-stage gate in which the model lists the information units a question needs, scores its support for each, and answers only when the mean score clears a threshold lowers the unconditional wrong-answer rate and raises accuracy on answered items for all eleven models tested, but most of the abstentions it produces fall on questions plain chain-of-thought answered correctly (about 69% at the primary operating point), so the gain is bought with lost coverage rather than with selective self-knowledge.Saying it does not know · reviewed by AI
- For multimodal LLMs, a behavioral perturbation signature (answer/confidence change under visual degradation, blank-image replacement, and grounding/relation probes) predicts which failure family an error belongs to far better than scalar confidence, yet folding the same signature into confidence as a direct penalty makes answer-versus-abstain ranking worse than scalar confidence alone.Saying it does not know · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: On the same datasets, answers after concept removal show the same or greater dependence on unmentioned factors as answers from standard prompting under both faithfulness metrics.