across-30-open-llama-mistral-qwen-models-8-classification CLM-0541Across 30 open LLaMA/Mistral/Qwen models and 8 classification tasks, a model's self-reported numeric confidence tracks its own logit-based confidence only weakly at the instance level (mean Pearson r about 0.135) even though setting-level averages of the two correlate moderately, and instruction-tuned models show larger linguistic-vs-internal gaps and worse calibration error than base models.
Capability: Saying it does not know
Observed on
Open-weight base and instruction-tuned models from three families; classification tasks with a fixed candidate set where logits over options are extractable, plus two generation ta.
Sources
- Measured over 30 models x 10 tasks with three axes (Pearson/Spearman, Euclidean distance, ECE) and paired t-tests; ECE difference significant (p=2.5e-8), correlation difference not (p=0.054). Regression attributes most variation to confidence-score dispersion and mean rather than model metadata. Logit softmax over options is a proxy, not ground-truth uncertainty, so the result is cross-channel fidelity plus a separate correctness-calibration measure; confidence elicited in the same turn as the answer, not post-hoc.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For RLHF-tuned chat models the raw conditional probabilities are consistently overconfident, and a confidence the model states in words or as a number in its output is better calibrated, often halving expected calibration error on TriviaQA, SciQ and TruthfulQA; asking the model to list several candidate answers before giving confidences improves it further, while chain-of-thought does not.Saying it does not know · reviewed by AI
- Open-weight instruction-tuned models, told in plain language to not think about a concept while transcribing a sentence, suppress that concept's residual-stream projection enough to evade linear probes, the Jacobian lens, activation oracles and natural-language autoencoders, with no fine-tuning or prompt search — detection at a 1% false-positive budget falls to roughly 5-21% under suppression while engagement is caught near ceiling.Strategic deception and detecting it · reviewed by AI
- Large pretrained models are well calibrated on multiple-choice and true/false questions when answer options are shown as lettered choices, and can predict the probability that their own sampled answer is correct (P(True)) or that they know the answer at all (P(IK)); calibration improves with model size and few-shot prompting, self-evaluation improves when the model sees several of its own samples first, and P(IK) generalizes only partly to new tasks, where its calibration degrades.Saying it does not know · reviewed by AI
- Instruction tuning that forces an answer to every question teaches models to fabricate when the question lies outside their parametric knowledge; splitting the tuning data by whether the base model already answers correctly and appending "I am unsure" to the rest (refusal-aware tuning) raises average precision on known-versus-unknown separation over vanilla tuning, transfers to out-of-domain datasets, and yields better calibrated confidence than post-hoc uncertainty scoring.Saying it does not know · reviewed by AI
- For small open-weight Qwen2.5 models (1.5B-3B) doing multi-step math reasoning, features derived from the spread of attention entropy across layers at each generation step predict whether the final answer is wrong better than output-logit confidence features, with the margin larger on harder problems (+0.0093 AUC on GSM8K, +0.0759 on MATH-500 at 3B).Stating false facts confidently · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: Measuring per-instance correlation between verbalized score and option-token probability across a comparable model set and finding consistently strong association (e.g. r above 0.5), or finding instruction-tuned models have lower linguistic ECE than their base counterparts. Drafted stance toward rlhf-tuned-chat-models-raw-conditional-probabilities-consistently-overconfident-confidence: contests -- This paper finds instruction-tuned models' verbalized confidence has significantly larger expected calibration error than base models' (0.287 vs 0.191) and only weak instance-level agreement with internal probabilities, cutting against the claim that verbalized confidence is better calibrated than conditional probabilities. Automatic check flagged: figures not in the source: 0.054.