dair-2026-modularrsi-three-defects SRC-0375 · post

Impressive paper on building recursive self-improving agent harnesses

DAIR.AI

Created: 2026-09-21 · Ingested: 2026-09-21

https://x.com/dair_ai/status/2101835306953736396

A summary post of ModularRSI (arXiv 2609.14857) that names the three defects the paper attributes to how harnesses get improved and the fix for each: evolving against the evaluation benchmark (fix: evolve on disjoint tasks); updating from single trajectories (fix: contrast success and failure on the same task, aggregate across tasks); a monolithic harness that cannot localize a problem (fix: five separately evolved modules). Secondary to the paper.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsEvolving an agent harness on 2,000 executable tasks disjoint from the evaluation benchmarks, by contrasting successful and failed trajectories on the same task to find recurring deficiencies and evolving five harness modules (agent loop, tool use, observation management, context management, completion detection) separately before integrating them, raised DeepSeek-V4-Flash from 47.6 to 52.4 on TerminalBench 2.0 and 73.4 to 76.5 on SWE-Bench Verified, transferred across domains and to other backbones frozen (GLM-5.2 59.6 to 61.8, MiniMax-2.5 41.6 to 44.9), while evolving the whole harness at once or without modules fell below the unevolved baseline (46.4 and 44.2).