post-training-small-llm-qwen2-5-0-5b-3b-produce-weekly CLM-0467mechanismsingle paperReviewed by AI
When post-training a small LLM (Qwen2.5 0.5B-3B) to produce weekly asset-return forecasts from news, rewarding rank correlation of a full listwise asset ordering gives a dense learning signal where exact-match GRPO gives almost none (all sampled actions scoring zero on ~35% of steps), and yields higher Spearman correlation and backtest Sharpe than the SFT checkpoint, standard GRPO, and prompted commercial models.
Capability: Predicting future events
Observed on
15 liquid ETFs, weekly horizon, 2010-2019 train / 2020-2025 test, Qwen2.5 base and instruct up to 3B; absolute rank correlations are tiny (~0.02) and evaluation may be contaminated.
Sources
- Validated by: Compared against equal weight, mean-variance, momentum, three commercial LLMs, and same-checkpoint SFT/standard-GRPO/NDCG@3/Kendall reward variants on Spearman, Kendall, NDCG@k, annualized return, Sharpe (0.636 best), and max drawdown, averaged over 5 trials; plus news-removal ablation (Sharpe 0.636 to 0.045) and a sentiment-inversion faithfulness check.. Single paper, one asset universe, one 2020-2025 test window of 314 weekly samples. Reward-metric ablation from the same SFT checkpoint isolates the reward design. [truncated]
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In GRPO-style RL post-training of 7B multimodal reasoning models, composing each optimization group from rollouts stratified by length-normalized sequence surprisal and outcome (confident-correct anchors, confident-wrong negatives, low-confidence successes) within the same step keeps path entropy from collapsing and avoids the late-training performance decline GRPO and DAPO show after a few hundred steps.Retaining skills and safety after fine-tuning · reviewed by AI
- When injecting new or counterfactual facts into a 3-4B instruct model by post-training, supervised fine-tuning on extracted QA pairs binds facts to their training question form — recall of the single fact stays reasonable while multi-source and inferential questions over the same facts collapse — and adding more SFT epochs does not close the gap, whereas mixed-policy RL that supplies the golden answer as an off-policy trajectory when on-policy rollouts fail recovers much of the generalization.Absorbing and flexibly using new knowledge · reviewed by AI
- For small instruction-tuned models (3-4B) answering closed-form true/false questions under a stated user opinion, scoring each sampled response by how much more common its answer is than the group itself predicted, and using that score as the GRPO reward, reduces answer flips under user pressure and raises accuracy without any labels, roughly matching label-supervised desycophancy fine-tuning at higher compute cost.Telling the user what they want to hear · reviewed by AI
- For 2B-8B video multimodal models, adding a small fraction of time-reversed video rollouts into each GRPO group (e.g. 1 reversed out of 7) raises accuracy on action and temporal-reasoning video QA over standard GRPO, while raising the reversed share further (2 or 3 of 7) gives back the gain.Reasoning about time in video · reviewed by AI
- In GRPO-style RLVR on math for 7B models, injecting partial reasoning prefixes from a small out-of-family model (Gemma-2-2B, LLaMA-3.2-1B) into a fraction of rollouts keeps policy entropy higher early in training and raises pass@k, with gains growing as k rises; the effect comes from distributional unfamiliarity rather than prefix quality, since prefixes from same-family models of equal or higher quality give no gain over vanilla GRPO.Retaining skills and safety after fine-tuning · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: A replication on a different asset universe or period where rank-correlation-reward training gives no Sharpe or Spearman gain over the SFT checkpoint, or where exact-match/numeric-target GRPO matches it.