agent-generates-whole-python-repository-natural-language-requirements-no-given CLM-0544When an agent generates a whole Python repository from natural-language requirements with no given architecture, keeping the repository architecture as a mutable state that is re-evolved before code generation yields higher functionality coverage and test pass rate than executing a single up-front planning graph, and removing the evolution step is the largest single ablation loss.
Capability: Generating and editing working code
Observed on
Six real-world Python repositories from RepoCraft, GPT-5 mini and DeepSeek V3.2 backbones, with all methods sharing the same downstream TDD code-generation and repair scaffold..
Sources
- Measured on three of six repositories reported in the main table (others in supplementary), two backbones, three averaged runs, cross-model evaluation to limit judge bias; baselines share the generation pipeline so architecture is isolated, but metric thresholds were tuned on held-out repos and all quality scores come from LLM-based matching plus adapted tests.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- When an agent builds a whole Python repository from natural-language requirements alone, refining component boundaries with explicit cohesion and coupling thresholds until no split or merge is triggered produces higher functionality coverage and test pass rate than letting the LLM decide restructuring actions for a fixed budget of rounds, because unconstrained LLM-decided actions keep over-decomposing past the point of a coherent modular structure.Generating and editing working code · reviewed by AI
- For single-agent procedural tasks where the relevant state fits a fixed schema known in advance, replacing the appended conversation transcript with a structured execution state that the model rewrites each step — discarding the reasoning trace once the state patch is validated — holds accuracy flat as the horizon grows while history-appending runtimes degrade, and the gain comes from the structure rather than the shorter prompt.Completing tasks that take many steps or many hours · reviewed by AI
- For LLM-driven heuristic discovery on combinatorial optimization tasks (TSP, CVRP, bi-objective and multi-component variants), letting an agent decide when to read, edit, diagnose and evaluate code while persisting each evaluation outcome as a retrievable experience card reaches better final heuristic quality than fixed evolutionary pipelines (EoH, ReEvo, FunSearch, MCTS-AHD, MEoH, MOTIF) at a fraction of their evaluation budget, and re-running the same task warm-started from the previous run's best code improves further round after round under a fixed budget.Fixing its own mistakes · reviewed by AI
- When an agent must work over a very large tool catalogue, wrapping each tool in a natural-language interface that resolves the API schema internally — so raw schemas are retrieved on demand rather than enumerated in the model's prompt — contributes more to multi-step tool- call success than the surrounding planner, router, or verifier stages.Using the tools it is given · reviewed by AI
- For LLM agents that must compose and then execute domain tool workflows, training separate adapters for the two roles splits the gains cleanly: supervised fine-tuning of the planner on reference tool sequences mainly improves workflow-order fidelity, while node-level RL on argument-match rewards mainly improves tool-argument grounding and final-answer quality, and combining both beats either on a 9B backbone.Planning a sequence of actions before acting · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A run holding the downstream generation scaffold fixed where the architecture-evolution loop gives no coverage or pass-rate gain over a one-shot planning graph, or where removing the evolution step costs less than removing other components. Drafted stance toward agent-builds-whole-python-repository-natural-language-requirements-alone-refining: supports -- Same setup and finding: metric-guided boundary evolution to convergence beats both one-shot planning and fixed-budget LLM-decided restructuring on coverage and pass rate.