Models complied with insecure completions a large fraction of the time, and more capable models were more likely to suggest insecure code.

Capability: Writing secure code and dependencies

Sources

supportsPurple Llama CyberSecEval: A Secure Coding Benchmark for Language Models
Models complied with insecure completions a large fraction of the time, and more capable models were more likely to suggest insecure code.

Reviewed by AI and Russ MillerLast checked: 2026-09-03Evidence activity

Either way, bring evidence: a link, or what you saw.

Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.

Related claims

Users with an AI assistant wrote less secure code and were more confident it was secure.Writing secure code and dependencies
On code prompts that are unsatisfiable by design, open-weight code and reasoning models refuse based on how suspicious the request looks rather than how deeply impossible it is: they refuse famous theoretical impossibilities far more often than requests for plausible-sounding nonexistent packages, and the stronger models in the set close the gap on theory while barely improving on fabricated ecosystem entities.Stating false facts confidently · reviewed by AI
Roughly forty percent of Copilot completions in security-relevant scenarios were vulnerable.Writing secure code and dependencies
On verifiable instructions such as length and format constraints, strong models still fail a meaningful share, and failures grow when several constraints apply at once.Following an unfamiliar procedure
Auditing AI-generated code with static smell detection at code, structure and architecture level, from single-file tasks up to agent-built systems, more capable models produced more bloated and more coupled code, code volume was a near-perfect predictor of structural decay, and neither functional correctness nor more detailed prompting reduced it, so passing tests does not indicate maintainable design.Making design decisions that hold up as a system grows · reviewed by AI