new-knowledge-set-interconnected-fictional-future-events-rather-than CLM-0458observationsingle paperReviewed by AI
When new knowledge is a set of interconnected fictional future events rather than isolated fact edits, updating 4-8B instruct models by mid-training on teacher-generated long-form articles plus preference tuning (preferring reasoned answers over outdated, hallucinated, or needlessly abstaining ones) beats graph-retrieval and in-context editing baselines on multi-hop and causal questions, while remaining worse than in-context methods on exact numeric single-hop recall unless retrieved facts are also supplied.
Capability: Absorbing and flexibly using new knowledge
Observed on
LLaMA-3.1-8B, Olmo-3-7B and Gemma-3-4B instruct models; 20 to 1,536 inserted facts from a synthetic future-event knowledge graph; GPT-4.1 as the teacher generating the synthetic da.
Model cards: GPT-4
Sources
- Validated by: QA accuracy on the ParallelEvents benchmark (single-hop, multi-hop, causal) after inserting 542 facts: 73.67% (LLaMA-3.1), 64.92% (Olmo-3), 79.07% (Gemma-3) versus best in-context baseline IKE at 63.32/57.73/53.73%, graph-retrieval baselines at 17.09%, and LoRA/single-layer tuning at 12.02%; paired bootstrap significance; general-ability checks on MMLU-Pro, PopQA, IFEval, IFBench; portability check on MQuAKE-Remastered. [truncated]
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Knowledge-editing methods such as ROME and fine-tuning insert a stated fact reliably but do not carry its logical consequences: on multi-hop questions generated from knowledge-graph rules that the edit entails, accuracy is up to 24 points lower than on the directly edited fact, so an edited model can assert a new fact and still answer as if its implications never changed.Absorbing and flexibly using new knowledge · reviewed by AI
- A single fact edit exerts two coupled pressures on its knowledge neighbourhood, propagation to facts that should change and leakage into facts that should not, and planning the edit's target representations jointly under both constraints, with a gate that rejects risky plans before touching parameters, improved propagation and preservation on RippleEdits by at least 7% across backbones over methods that treat the two separately.Absorbing and flexibly using new knowledge · reviewed by AI
- When injecting new or counterfactual facts into a 3-4B instruct model by post-training, supervised fine-tuning on extracted QA pairs binds facts to their training question form — recall of the single fact stays reasonable while multi-source and inferential questions over the same facts collapse — and adding more SFT epochs does not close the gap, whereas mixed-policy RL that supplies the golden answer as an off-policy trajectory when on-policy rollouts fail recovers much of the generalization.Absorbing and flexibly using new knowledge · reviewed by AI
- In LLM fact-checking over supplied evidence, high veracity accuracy does not imply the verdict depends on that evidence: when the model's own cited sentences are iteratively removed, off-the-shelf models (GPT-4o-mini, Gemini-2.5-Flash-Lite, Qwen-2.5-32B, Llama-3.1-8B) keep their SUPPORT/REFUTE labels at a non-zero plateau instead of abstaining, and training with paired supervision that requires NOT_ENOUGH_INFO on the evidence-ablated input removes most of that persistence without losing accuracy.Checking claims against evidence · reviewed by AI
- Unlearning methods that make a model answer "I don't know" about a forget set can leave the knowledge in place: the strongest IDK-style fine-tuning method still answered those same questions accurately when probed in a different format, and across nine unlearning methods on three model families none met a joint standard of utility and honesty on retained knowledge plus consistent, acknowledged forgetting; aligning representations after feature randomization (ReVa) roughly doubled the rejection rate that survives a second conversational turn.Retaining skills and safety after fine-tuning · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: Fine-tuning baselines trained on the same synthetic articles and QA pairs matching or beating the preference-tuned pipeline on causal and multi-hop accuracy, or in-context retrieval methods matching it once given comparable fact retrieval.