retrieval-augmented-personalization-frozen-8-9b-generator-prepending-more-retrieved-user-history CLM-0219In retrieval-augmented personalization with a frozen 8-9B generator, prepending more retrieved user-history records does not monotonically improve quality — on the six LaMP tasks a fixed top-k profile peaked at k=5 and lost 0.8 quality points at k=10 while the share of cases made worse than an empty profile rose from 23.2% to 31.7% — so a learned per-request stopping decision beats any single fixed k.
Capability: Remembering across sessions
Observed on
LaMP's six personalization tasks (classification, ordinal rating, short text generation), Contriever candidate pool of at most 20 records, frozen Qwen3.5-9B and Llama-3.1-8B-Instru.
Sources
- supportsLess Is Personal: Learning Minimal Sufficient User Profiles for Personalized Language ModelsMeasured on LaMP with two frozen generators against eight retrieval-augmented baselines; ENOUGH is first in all 24 task-metric cells with paired t-tests, and a fixed-length grid plus forced-continuation-past-stop diagnostics (beneficial share 18.0% to 9.9%, harmful 16.9% to 35.9% from +1 to +3 records) support the non-monotonicity. Ablations isolate long-horizon supervision, adaptive stopping, specificity and token-cost terms. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For agentic retrieval-augmented multi-hop QA with 7B-class policy models, scoring each intermediate step for both logical validity and grounding in the retrieved passages — rather than scoring steps by the final answer — yields higher EM/F1 than outcome-reward and final-answer-based process-reward training, because it removes preference pairs where flawed retrieval happened to reach the right answer.Carrying a chain of inference through to its conclusion · reviewed by AI
- When the preference needed to answer a query lies in a different concept space from the cues in a user's dialogue history (e.g. inferring an emotion-regulation strategy from personality cues), retrieval-based and profile-summarization personalization methods barely beat a no-history baseline, and dumping the full history into the prompt does worse than no history at all.Remembering across sessions · reviewed by AI
- On multi-year authentic personal photo/video archives turned into memory streams, frontier multimodal models and structured memory systems recall facts and infer persona far better than they predict a user's future choices: predicting or recommending in novel contexts caps around 59 on a 0-100 scale for every system tested, and compressing the history ~70x into a memory system costs nothing on that tier while costing a lot on factual recall, so predictive personalization is limited by preference representation rather than by retrieval or context length.Remembering across sessions · reviewed by AI
- When a user profile of stated attributes and preferences is placed in context, models agree with the user far more than with no profile — across 13 open and closed models the profile, not retrieved conversation memory, is the dominant driver, and inverting the stated preference flips almost all responses to the opposite side.Telling the user what they want to hear · reviewed by AI
- For small instruction-tuned models (3-4B) answering closed-form true/false questions under a stated user opinion, scoring each sampled response by how much more common its answer is than the group itself predicted, and using that score as the GRPO reward, reduces answer flips under user pressure and raises accuracy without any labels, roughly matching label-supervised desycophancy fine-tuning at higher compute cost.Telling the user what they want to hear · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A fixed-k profile whose quality rises monotonically with k on the same tasks and pools, or a learned adaptive-stopping controller that fails to beat the best development-selected fixed k. Proposed technique, not catalogued: Learned adaptive-length user-profile controller with an explicit stop action.