accurate-retrieved-context-supplied-every-turn-three-turn-dialogue-model CLM-0274When accurate retrieved context is supplied at every turn of a three-turn dialogue, model scores still fall as each turn adds a condition or constraint on how the retrieved material must be used, and models that score alike on plain multi-turn chat diverge on retrieval-grounded turns, so using retrieval well is a distinct competence from chatting well.
Capability: Using what was retrieved, and only that
Observed on
RAD-Bench, 89 three-turn samples over six scenarios (synthesis and reasoning over retrieved text), 2024; scored 1-10 by an LLM judge against references; open and closed models of 2.
Sources
- Small benchmark (89 samples) judged by a model; the per-turn decline and the chat-versus-retrieval divergence are the two findings, both shown in figures rather than with confidence intervals.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For agentic retrieval-augmented multi-hop QA with 7B-class policy models, scoring each intermediate step for both logical validity and grounding in the retrieved passages — rather than scoring steps by the final answer — yields higher EM/F1 than outcome-reward and final-answer-based process-reward training, because it removes preference pairs where flawed retrieval happened to reach the right answer.Carrying a chain of inference through to its conclusion · reviewed by AI
- For conversational long-term memory with 8-14B backbones and plain top-K dense retrieval, writing each dialogue span into several complementary low-overlap semantic views before indexing retrieves gold evidence for more queries than structured extraction into a single view, and the advantage is largest at small retrieval budgets.Remembering across sessions · reviewed by AI
- For RLHF-tuned chat models the raw conditional probabilities are consistently overconfident, and a confidence the model states in words or as a number in its output is better calibrated, often halving expected calibration error on TriviaQA, SciQ and TruthfulQA; asking the model to list several candidate answers before giving confidences improves it further, while chain-of-thought does not.Saying it does not know · reviewed by AI
- When the preference needed to answer a query lies in a different concept space from the cues in a user's dialogue history (e.g. inferring an emotion-regulation strategy from personality cues), retrieval-based and profile-summarization personalization methods barely beat a no-history baseline, and dumping the full history into the prompt does worse than no history at all.Remembering across sessions · reviewed by AI
- For spoken language models answering questions over long dialogue histories, converting the history to transcripts beats keeping it as audio when the answer is a lexical detail (exact slot values across dense turns: mean 71.3% vs 42.6%), but loses when the answer depends on how something was said, where audio history wins for six of seven models and by a larger margin once lexical emotion cues are removed.Understanding speech and audio · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: Current models showing flat scores across turns on the same benchmark, or a re-judging with humans that removes the turn-wise decline.