declared-dependencies-are-not-the-set-the-code-needs CLM-0038observationsingle paperReviewed by AI and Russ Miller
Checking that every package an agent names actually exists catches the invented ones but not the missing ones: what agents declare is routinely a fraction of what their code imports at runtime, so generated projects fail in a clean environment even when every declared package is real. Existence is a necessary check, not a sufficient one.
Capability: Writing secure code and dependencies · Security, Coding agent
Observed on
2025, Claude Code / OpenAI Codex / Gemini coding agents.
Applies when
- Effect on the capability:
- helps, narrowly
- Needs:
- an environment where code can run
- Cost:
- low
- Fails when:
- The missing package is one the agent never declared. Existence checks catch invented packages, not omitted ones, so the project still fails in a clean environment.
Sources
- Across 300 generated projects run in a clean environment with only the declared dependencies, 68.3% executed out of the box (Python 89.2%, Java 44.0%), and actual runtime dependencies averaged 13.5x the declared set.
- The other half of the same surface: packages that do not exist at all, which is what the existence check is aimed at. Together they bound the technique — it addresses one failure mode of two.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
Notes
Filed as an observation, not a mechanism: the numbers are tied to three specific 2025 agents and will move. The durable part — that a declared manifest and an import graph are different objects — is the reason the check has to compare against what the code actually imports rather than only against the registry.