arxiv-2311-09677 SRC-0308 · paper

R-Tuning: Instructing Large Language Models to Say `I Don't Know'

Created: 2023-11-16 · Ingested: 2026-09-19

Large language models (LLMs) have revolutionized numerous domains with their impressive performance but still face their challenges. A predominant issue is the propensity for these models to generate non-existent facts, a concern termed hallucination. Our research is motivated by the observation that previous instruction tuning methods force the model to complete a sentence no matter whether the model knows the knowledge or not. When the question is out of the parametric knowledge, it will try to make up something and fail to indicate when it lacks knowledge. In this paper, we present a new ap

In brief

Fine-tuning a model on its own errors, labelled with "I am unsure", teaches refusal as a transferable skill and also improves answer accuracy on the questions it still attempts. R-Tuning splits an instruction-tuning set into certain data (model's pre-tuning prediction matches the label) and uncertain data, appends "I am sure"/"I am unsure" to each, then fine-tunes normally.

Tested on OpenLLaMA-3B, LLaMA-7B and LLaMA-13B across 9 datasets, single-task on ParaRel and MMLU and multi-task on 5 datasets with HaluEval-QA held out, scored by Average Precision over confidence-ranked answers.

AP gains over vanilla instruction tuning grow with size: MMLU in-domain 68.87 vs 51.93 at 13B but 59.05 vs 58.16 at 7B; ParaRel OOD 77.30 vs 64.12 at 13B, and 74.61 vs 78.08 at 7B, the one reversal. On unanswerable questions, refusal rates reach 95.90–99.18 for R-Tuning versus 2.07–34.79 for vanilla, with baselines explicitly prompted that refusal was allowed.

Comparisons are matched against vanilla tuning on the same data, and an unsupervised variant (10 samples, entropy split) plus a test-time uncertainty-filtering baseline (Vanilla-C) are both included; R-Tuning-U usually beats Vanilla-C, e.g. 64.69 vs 48.34 on MMLU ID at 7B. All results are self-reported on 1 model family, binary confidence only, no chat-tuned or larger models.

If you are building abstention behaviour, the training-time route looks cheaper and better calibrated than sampling at test time.

Written from the abstract by claude-opus-5 on 2026-10-17, with every figure checked against it. Not a substitute for the paper.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsInstruction tuning that forces an answer to every question teaches models to fabricate when the question lies outside their parametric knowledge; splitting the tuning data by whether the base model already answers correctly and appending "I am unsure" to the rest (refusal-aware tuning) raises average precision on known-versus-unknown separation over vanilla tuning, transfers to out-of-domain datasets, and yields better calibrated confidence than post-hoc uncertainty scoring.