auditing-ai-generated-code-static-smell-detection-code-structure-architecture CLM-0293Auditing AI-generated code with static smell detection at code, structure and architecture level, from single-file tasks up to agent-built systems, more capable models produced more bloated and more coupled code, code volume was a near-perfect predictor of structural decay, and neither functional correctness nor more detailed prompting reduced it, so passing tests does not indicate maintainable design.
Capability: Making design decisions that hold up as a system grows
Observed on
Python solutions from several LLMs and agent frameworks, PyExamine smell detection, zero- and few-shot, 2026; Spearman correlations..
Sources
- supportsAI-Generated Smells: An Analysis of Code and Architecture in LLM and Agent-Driven Development'Near-perfect' is the authors' description of a correlation; the reasoning-complexity trade-off is asserted across models without a controlled capability axis.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- When a benign GitHub issue is augmented with a plausible-sounding maintenance requirement that specifies a helper reading an environment variable into a dangerous sink, repository-level APR agents produce patches that fix the original bug and pass the project test suite while containing the requested vulnerability in roughly half of tasks, and the strongest instruction-following coding models are more susceptible, not less.Writing secure code and dependencies · reviewed by AI
- A linear probe on a tool-calling model's hidden state at the last generated token detects incorrect tool-calls — including wrong-but-well-typed argument values that parsers and logs do not catch — and works better on larger models and at middle-to-late layers than at the final layer.Using the tools it is given · reviewed by AI
- On code prompts that are unsatisfiable by design, open-weight code and reasoning models refuse based on how suspicious the request looks rather than how deeply impossible it is: they refuse famous theoretical impossibilities far more often than requests for plausible-sounding nonexistent packages, and the stronger models in the set close the gap on theory while barely improving on fabricated ecosystem entities.Stating false facts confidently · reviewed by AI
- When a model is asked to audit many documents for planted errors in a single large-batch prompt, recall of known contaminants collapses and the model reports confident fabricated findings — invented absurd contaminants that appear in no document — rather than signalling incomplete processing.Stating false facts confidently · reviewed by AI
- For single-step tool-calling agents, near-ceiling accuracy on unperturbed calls does not predict robustness to perturbation, and corrupting or removing the tool's returned evidence hurts far more than perturbing tool names, descriptions, or the user request.Using the tools it is given · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: Models or prompting regimes where correctness and smell counts decouple, or where more capable models produce leaner code.