small-open-weight-qwen2-5-models-1-5b-3b-doing-multi-step CLM-0490For small open-weight Qwen2.5 models (1.5B-3B) doing multi-step math reasoning, features derived from the spread of attention entropy across layers at each generation step predict whether the final answer is wrong better than output-logit confidence features, with the margin larger on harder problems (+0.0093 AUC on GSM8K, +0.0759 on MATH-500 at 3B).
Capability: Stating false facts confidently
Observed on
White-box access to all attention layers; only Qwen2.5 1.5B and 3B; only GSM8K and MATH-500; baseline is output-entropy/probability features only, with no sampling-based baselines .
Sources
- Validated by: ROC-AUC under repeated stratified 5-fold cross-validation (10 repeats), against output-logit baseline features, with 95% CIs and one-sided Wilcoxon signed-rank tests; gains of +0.0093 to +0.0759 AUC.. Measured on two benchmarks and two model sizes with cross-validated logistic regression and significance tests, but absolute AUCs are low (0.58-0.70), sample sizes small, no sampling-based baseline, and scale and task difficulty are confounded. Step-parsing coverage was incomplete on MATH-500, possibly a selection effect.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For a domain-specific language essentially absent from pretraining corpora (FORM symbolic algebra), QLoRA fine-tuning an 8B model on a few thousand examples validated by executing them through the language's own interpreter beats frontier open-weight models of up to 756B parameters given syntax documentation in context, on execution rate and strict output match for the larger benchmarks, while general reasoning scores drop under 3 points.Generating and editing working code · reviewed by AI
- For small instruction-tuned models (3-4B) answering closed-form true/false questions under a stated user opinion, scoring each sampled response by how much more common its answer is than the group itself predicted, and using that score as the GRPO reward, reduces answer flips under user pressure and raises accuracy without any labels, roughly matching label-supervised desycophancy fine-tuning at higher compute cost.Telling the user what they want to hear · reviewed by AI
- In 7-8B open-weight instruct models (Llama-3.1, Mistral-v0.3, Qwen2.5), the drop in a token's probability when a contiguous mid-layer MLP span is zero-ablated is near zero for function words, elevated for knowledge-bearing tokens, and abnormally spiked for hallucinated tokens, so this internal attribution signal — approximable by a small probe from one forward pass — can be used in beam scoring to raise factuality on open-ended and long-form generation, while leaving strict short-answer EM/F1 unchanged or worse.Stating false facts confidently · reviewed by AI
- When 7-8B open-weight instruct models are fully fine-tuned on a single domain dataset, confining gradient updates to FFN neurons that activate stably across the training set and adding an MSE loss that pulls activations back toward pre-computed activation anchors retains far more general-benchmark ability (MMLU, BBH, TyDiQA) than full-parameter fine-tuning, by about 9.9 points on average, while also raising in-domain scores.Retaining skills and safety after fine-tuning · reviewed by AI
- For a same-family pair of reasoning and instruction-tuned 4B models, learning one interpolation ratio per module (attention block, FFN block, embedding, final norm) with a reward that scores correctness and ranks correct responses by length cuts average generated tokens by about 27% relative to the reasoning model while holding average accuracy, and beats fixed global-ratio merging and CMA-ES search over the same ratios.Cutting to the chase · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: On the same or other open-weight models, attention-dispersion features fail to beat output-logit baselines in AUC, or the gain disappears when compared against sampling-based baselines such as semantic entropy, or on non-math tasks.