arxiv-2609-04343 SRC-0203 · paperA Removal Based Approach to Improve LLM Faithfulness at Test-Time
Qinglan Luo, S M A Nahian, John Guttag, S. Mazdak Abulnaga, Katie Matton
Created: 2026-09-03 · Ingested: 2026-09-19
https://arxiv.org/abs/2609.04343(opens in a new tab)Large language models (LLMs) are increasingly used for consequential decisions, making their explanations an important tool for auditing model behavior. Unfortunately, these explanations can be unfaithful, failing to reflect the actual reasoning underlying the model's decisions. We consider a setting in which an LLM provides both an answer and an explanation in response to a question. We identify two distinct dimensions of unfaithful explanations: incompleteness, meaning that the explanation omits factors that influence the answer, and unsoundness, meaning that the explanation cites factors th
In brief
Removing from the input any concept the model's own explanation did not cite, then re-querying on the reduced input, makes explanations more faithful at test time without touching weights. The intuition is that this strips hidden influences while leaving the credited concepts intact.
The paper separates 2 failure modes: incompleteness (the explanation omits factors that did influence the answer) and unsoundness (it cites factors that did not). Most existing test-time work targets unsoundness; this method targets incompleteness, where omitted factors are called hidden concepts. Evaluation spans 2 datasets, multiple model families, and 2 independent faithfulness metrics.
The reported result is directional only: faithfulness improves over both standard prompting and prompting that explicitly asks for faithful explanations. The source text available here gives no effect sizes, so the magnitude of the gain cannot be judged.
The design includes 2 prompting baselines and 2 metrics, which is the right shape for attribution, but no ablation of the removal step, no matched-compute comparison, and no independent replication are described. The illustrated motivating case is a stereotype-sensitive hiring question; whether the approach holds for long-form reasoning, agentic tasks, or accuracy-critical settings is untested.
If you audit model explanations, this is a cheap intervention worth trying, but you will need the full paper to size the benefit.
Written from the abstract by claude-opus-5 on 2026-09-18, with every figure checked against it. Not a substitute for the paper.
Referenced by
Claims in this catalog that draw on this source, and whether as support or counterpoint.
- supportsWhen an LLM answers a question and gives an explanation, removing from the input the concepts its explanation did not credit and re-querying on the reduced input raises measured explanation faithfulness over both standard prompting and prompts that ask for faithfulness, because the removal deletes the unmentioned influences rather than asking the model to report them.
- supportsWhen an LLM gives an answer plus an explanation, deleting from the input those concepts the explanation does not credit and re-querying on the reduced input raises measured faithfulness over both standard prompting and prompts that ask for faithful explanations, because the re-query removes the unmentioned factors that could have influenced the original answer.