arxiv-2608-25243 SRC-0239 · paper

From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection

Zhibo Hou, Fan Zhao, Zhiyu An, Wan Du

Created: 2026-08-26 · Ingested: 2026-09-19

Continual knowledge injection is essential for keeping large language models up-to-date in a fast-evolving world. Existing methods rely on supervised fine-tuning (SFT), which memorizes injected facts in their training format but fails to generalize across paraphrasing, document combinations, and reasoning. To address this, we propose Golden-GRPO Injection (GRIN), a three-stage self-learning framework for continual knowledge injection. Golden-GRPO is a mixed-policy reinforcement learning algorithm designed specifically for knowledge injection, which injects a golden answer to provide learning s

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsWhen injecting new or counterfactual facts into a 3-4B instruct model by post-training, supervised fine-tuning on extracted QA pairs binds facts to their training question form — recall of the single fact stays reasonable while multi-source and inferential questions over the same facts collapse — and adding more SFT epochs does not close the gap, whereas mixed-policy RL that supplies the golden answer as an off-policy trajectory when on-policy rollouts fail recovers much of the generalization.