roughly-forty-percent-copilot-completions-security-relevant-scenarios-weremechanismsingle paper
Roughly forty percent of Copilot completions in security-relevant scenarios were vulnerable.
Capability: Writing secure code and dependencies
Sources
- Roughly forty percent of Copilot completions in security-relevant scenarios were vulnerable.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.
Related claims
- Models complied with insecure completions a large fraction of the time, and more capable models were more likely to suggest insecure code.Writing secure code and dependencies