arxiv-2609-01600 SRC-0471 · paper

CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses?

Created: 2026-09-01 · Ingested: 2026-10-17

Dynamic agent harnesses let language models change the software that shapes their own execution. This flexibility brings a new reasoning burden: a local plugin change can propagate through dependencies and cleanup. We introduce CordisBench, a 1,200-question benchmark of this lifecycle reasoning. It combines a controlled formal setting with programs executed against Cordis, a runtime that manages component dependencies and cleanup, and asks models to identify affected components, predict state after a specified teardown order, determine which conditions hold under all or some orders, and choose

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsWhen a model must anticipate, without execution feedback, what a change to its own runtime harness components will do, naming which components a lifecycle change can affect stays near ceiling as the number of interacting effects grows from 2 to 32, while predicting the resulting state, reasoning across teardown orders, and choosing minimal reconfigurations degrade sharply (e.g. GPT-5.6 Luna's reachable-condition Jaccard 91.7% to 14.1%, executed reconfiguration success 62.5% to 25.0%).