Key facts.
- In the seminal study, GitHub Copilot produced a vulnerability in about40%of relevant completions across 89 scenarios drawn from MITRE's CWE Top 25 (Pearce, Ahmad, Tan, Dolan-Gavitt & Karri,"Asleep at the Keyboard?", arXiv:2108.09293, IEEE S&P 2022).
- In a controlled user study, participants with an AI assistant wrotesignificantly less secure codeon four of five tasks, yet weremore likely to believe their code was secure(Perry, Srivastava, Kumar & Boneh,"Do Users Write More Insecure Code with AI Assistants?", arXiv:2211.03622, ACM CCS 2023). On the SQL task, 36% of the AI group's code was vulnerable versus 7% of the control.
- It hasn't improved much with scale. Veracode's 2025 GenAI code security testing found roughly45%of AI-generated samples failed security checks, with no clear gain from newer or larger models (Veracode, 2025).
- An independent CSET evaluation of five models on C prompts found about48%of compilable snippets contained an insecure bug (CSET, Georgetown, November 2024).
If the model writes working code, why is it insecure?
Because "works" and "safe" are different targets, and the model was optimized for the first. Training data was public code. Public code is full of the exact insecure patterns it now reproduces: string-concatenated SQL, weak randomness, missing input validation, hard-coded credentials. No threat model. No sense of a trust boundary. No awareness that this particular input comes from an attacker. The most common completion is usually the insecure one, because that's what most code on the internet does. It compiles, passes the happy-path test, reads as confident and idiomatic. None of that's security.
The canonical example is the one the studies keep finding, building a SQL query by gluing strings together:
# What the assistant offers, fluent and wrong query = "SELECT * FROM users WHERE email = '" + email + "'" db.execute(query) # SQL injection: email is attacker-controlled# What it should be: a parameterized query db.execute("SELECT * FROM users WHERE email = ?", (email,))
Why is the confidence the dangerous part?
Because security depends on someone catching the flaw, and the assistant quietly removes the person most likely to catch it: a skeptical you. The Stanford study's sharpest finding wasn't just that the AI group wrote less secure code, but that they trusted it more, and rated their own insecure code as safe more often than the control group did. that's the failure compounding on itself. The tool produces a fluent answer, fluency reads as competence, competence reads as safety, and the review that would have caught the injection never happens because nobody felt the need. Less experienced developers, who most need the safety net, are the ones most likely to accept the suggestion uncritically. The speed is real and worth having. Same interface, two outputs: the speed you wanted, and a false signal that the code is safe.
Speed without the vulnerabilities: five practices
| Practice | What it catches |
|---|---|
| Treat AI output as an untrusted draft | Automation bias; restores the skeptical review the confidence removed |
| Run SAST / security scanners in CI | The recurring CWEs: injection, weak crypto, hard-coded secrets |
| Prompt with the security requirement | Default insecure patterns; asking for parameterized queries shifts the output |
| Mandatory human review of AI code | Logic and trust-boundary flaws scanners miss |
| Track which code came from AI | Lets you weight review and target audits where risk concentrates |
Near 40 percent of generated code ships insecure; a better model is fluent, not safe, and the breach waits late. (arXiv:2108.09293)
The pattern is that an AI assistant optimizes for code that looks right and runs, which isn't the same as code that's safe, and it pairs that gap with a confidence that suppresses the review that would close it. Scan the output, prompt for security explicitly, and keep a human signature on anything that ships. None of that's a smarter model, which in the benchmarks isn't measurably more secure. it's a layer that treats generated code as untrusted until verified, which is what VibeModel builds as the Pattern Intelligence Layer.
Frequently asked questions
Do newer, bigger models fix this?
Not in the data. Veracode's 2025 testing found roughly 45% of samples still failed security checks with no clear improvement from model size or recency. Functional ability rose; security didn't.
Is the 40% rate exaggerated?
It varies with benchmark and detection method, and the original authors flag their scenarios as simplified. But multiple independent studies (CSET ~48%, Veracode ~45%) land in the same range, so the direction is solid.
Should I stop using AI assistants?
No. The productivity is real. The fix is process: scan, prompt for security, and review AI output instead of trusting it, especially for anything touching input handling, auth, or crypto.