single-step-retrosynthesis-real-industrial-target-molecules-frontier-general-llms CLM-0637On single-step retrosynthesis for real industrial target molecules, frontier general LLMs produce chemically infeasible top-1 routes (unresolved selectivity or no mechanistic basis) more often than specialized retrosynthesis models, even though their reference-matching recall is comparable, so disconnection fluency does not carry selectivity awareness.
Capability: Stating false facts confidently
Observed on
350 internal industrial targets, single-step prediction, top-1 route inspected by expert chemists; three frontier LLMs vs seven specialized single-step models..
Sources
- Validated by: Expert-annotated top-1 chemical hallucination rate on 350 industrial targets fell from 14.3-27.4% to 0-6.9% across ten backbones, and top-1 recall rose from 53.1% (plain ensemble) to 74.3%; but the post-rerank hallucination figure is measured by the same verifier that does the reranking, so it is partly definitional, and no independent held-out verifier or wet-lab check is reported.. Measured on one company-internal 350-molecule benchmark with expert chemist annotation; [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For long-horizon LLM agents in environments where an erroneous action changes world state, restoring an earlier checkpoint and carrying forward a structured reflection memory that excludes branch-local state claims beats either intervention alone, because reflection alone cannot undo environmental damage and rollback alone re-explores the same failure.Completing tasks that take many steps or many hours · reviewed by AI
- In LLM multi-agent systems that reflect after failure, restricting reflection to the single agent responsible for the first incorrect step — located by a trained step-level correctness scorer and paired with a counterfactually generated corrected step — raises task success more than having every agent reflect on the trajectory, because reflections written by agents that behaved correctly enter their memory as misleading guidance for later tasks.Getting more from several models than from one · reviewed by AI
- In multi-agent LLM pipelines over reasoning and code tasks with a pool of 3B-32B open-weight backbones, conditioning each per-step routing decision on a learned filtered memory rather than the full accumulated trajectory leaves accuracy about the same while cutting inference cost by roughly 40-50%, so most of the accumulated history is cost without accuracy value.Getting more from several models than from one · reviewed by AI
- For LLM agents on dependent multi-step tasks, success decays geometrically with the number of steps under a single per-step reliability parameter that rises with model scale but stays below 1, so long horizons eventually collapse even for the strongest deployed systems — and the decay tracks step count, not context length, since bounding the context window steepens the decay rather than easing it.Completing tasks that take many steps or many hours · reviewed by AI
- For long-horizon embodied task planning where a symbolic model defines action preconditions and effects, correcting a violated plan by edits the action model itself implies (inserting the missing navigate or open) raises plan success more per LLM call than returning the violation to the LLM for replanning, and the gain persists for frontier models as well as compact ones.Planning a sequence of actions before acting · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: On an independent industrial retrosynthesis set with blind expert annotation, frontier LLMs' top-1 infeasible-route rate at matched recall is equal to or below that of specialized single-step models.