Checking that every package an agent names actually exists catches the invented ones but not the missing ones: what agents declare is routinely a fraction of what their code imports at runtime, so generated projects fail in a clean environment even when every declared package is real. Existence is a necessary check, not a sufficient one.

Capability: Writing secure code and dependencies · Security, Coding agent

Observed on

2025, Claude Code / OpenAI Codex / Gemini coding agents.

Sources

supportsAI-Generated Code Is Not Reproducible (Yet): An Empirical Study of Dependency Gaps in LLM-Based Coding Agents
Across 300 generated projects run in a clean environment with only the declared dependencies, 68.3% executed out of the box (Python 89.2%, Java 44.0%), and actual runtime dependencies averaged 13.5x the declared set.
supportsWe Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs
The other half of the same surface: packages that do not exist at all, which is what the existence check is aimed at. Together they bound the technique — it addresses one failure mode of two.

Status: activeLast checked: 2026-09-04Evidence activity

Contest this claim

Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.

Related claims

Across many models and hundreds of thousands of samples, a substantial share of recommended packages did not exist, and the names repeated, making them exploitable.Writing secure code and dependencies

Notes

Filed as an observation, not a mechanism: the numbers are tied to three specific 2025 agents and will move. The durable part — that a declared manifest and an import graph are different objects — is the reason the check has to compare against what the code actually imports rather than only against the registry.