outside-mathematics-step-level-error-detection-both-process-reward-models CLM-0315observationsingle paperReviewed by AI
Outside mathematics, the step-level error detection of both process reward models and general LLMs is markedly weaker than on math benchmarks, and the two fail differently: PRMs miss knowledge-based errors while LLMs miss computational errors, across science and logic domains.
Capability: Spotting an error in a given artifact
Observed on
GR-Ben, two domains (science, logic) and nine subdomains, 22 PRMs and LLMs, 2026; human-annotated step errors with about 15% of items discarded in annotation..
Sources
- Benchmark from one group; the complementary-failure pattern is reported as a general finding without per-model detail here. Consistent with Hard2Verify's picture that verification lags generation.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For step-level verification of structured quantitative STEM reasoning, training a 7B process reward model only on hard negatives that are constructed to pass a deterministic symbolic verifier (swapped formula or unit-matched variable, with values recomputed so the math balances), and running the verifier as a hard filter before the PRM scores candidates, detects flawed steps better than monolithic PRMs that must judge arithmetic and logic together.Spotting an error in a given artifact · reviewed by AI
- On human-annotated step-level verification of frontier-model proofs to recent IMO and Putnam problems, most of 29 generative critics and process reward models do little better than chance at finding the first wrong step, a leading math PRM (Qwen2.5-Math-PRM-72B) reaches only about 37% balanced F1, and beyond a few standouts open-source verifiers lag closed models; verification is harder than generation for the same frontier models.Spotting an error in a given artifact · reviewed by AI
- Used to steer test-time compute, generative LLM judges (7B-70B) rerank candidate responses about as well as outcome reward models, are consistently worse than process reward models at step-level beam search, and their natural-language critiques, the one thing only a judge can produce, did not guide generators to better responses on math, code or instruction following.Judging another's work as well as a careful reviewer would · reviewed by AI
- When training an LLM with policy-gradient RL on tasks that decompose into many dependent sub-problems, rewarding each trajectory segment by how many reference reasoning points it newly reaches beats a sparse final-answer reward by a margin that widens with task horizon, and still learns on problem sets where the base policy never samples a success and sparse reward gives no gradient at all.Completing tasks that take many steps or many hours · reviewed by AI
- When a calculator-using model is trained with reinforcement learning on verifiable final-answer rewards for an arithmetic search task (Countdown), the gain lands almost entirely at low k — pass@1 rises sharply while pass@16 moves little or falls — because the update can only reinforce correct trajectories the starting policy already samples, and prompt groups with no correct sample supply no gradient.Using the tools it is given · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A PRM trained on general-domain process labels matching its math-domain detection rate on GR-Ben.