The pull request compiles. The demo runs. The code reads cleaner than anything your team writes by hand on a Friday afternoon. It's also wrong — and wrong in a way no engineer on your payroll would have been wrong. Which is why you can't review it like human work: you have to test AI-generated code against the way it actually fails.
Key takeaways
To test AI-generated code, verify it against anchors outside the model: acceptance tests derived from your API contracts, registry checks on every new dependency, OWASP-class security scanning in CI, a regression suite that grows with every release, and human review reserved for auth, payments, and user data.
Let's get one thing out of the way: this is not an anti-AI post. We ship with Claude and Copilot every day, and statistically so do you: DORA's 2025 survey of roughly 5,000 practitioners puts AI adoption at 90% [C10]. Sundar Pichai told investors in late 2024 that more than a quarter of all new code at Google is generated by AI, "then reviewed and accepted by engineers" [C11]. Hold on to that second clause. Google didn't announce AI-generated code; it announced AI-generated code plus a verification step.
The step matters because AI code fails differently than human code. A tired human writes code that looks tired: the off-by-one, the missing null check, the half-finished error handler. AI output has none of those smells. It fails confidently.
Worse, it recalibrates your judgment in the wrong direction. An early Stanford study (CCS '23, codex-era assistants) found participants with an AI assistant wrote significantly less secure code and were more likely to believe they'd written secure code [C13]. The overconfidence effect showed up in the very first generation of tools. It hasn't aged out either: METR ran a randomized trial in 2025 with 16 experienced open-source developers across 246 real tasks on early-2025 tooling. AI use increased completion time by 19%. The same developers estimated they'd been 20% faster [C14].
Read those together and the conclusion is uncomfortable: "it looks right to me" is precisely the instrument this class of failure defeats. Verification has to move from vibes to external anchors.
AI-generated code fails in three verified ways: security vulnerabilities (45% of samples fail OWASP security tests [C1]), hallucinated dependencies (19.7% of recommended packages don't exist [C5]), and plausible-but-wrong logic (AI-co-authored PRs carry roughly 1.7x the issues of human-authored ones [C12]).
Start with security, because it's the flattest line in software right now. Veracode ran 80 coding tasks across more than 100 LLMs and found 45% of the generated samples failed security tests against OWASP Top 10 vulnerability classes [C1][C3]. Java was worst at a 72% failure rate, and cross-site scripting was missed in 86% of the samples where it applied [C2]. Yes, Veracode sells security scanning, so discount the framing, but the longitudinal number is hard to argue with: across 150+ models over two years, security pass rates hovered between 45% and 55% regardless of model generation or release date. As the report puts it: "Two years of 'revolutionary' model releases have moved the security needle from approximately 55% to… approximately 55%" [C3]. Newer models did improve — at writing functional code. Security performance stayed flat [C4].
The second failure mode has no human equivalent at all. In a USENIX Security '25 study of 576,000 code samples from 16 LLMs, 19.7% of the packages the models recommended did not exist: 205,474 unique invented names, with hallucination rates of at least 5.2% for commercial models and 21.7% for open-source ones [C5]. Your developers pick bad dependencies sometimes. They do not pick fictional ones. And these fictions are stable: 43% of hallucinated package names repeated every single time across ten reruns [C6]. That predictability is what turns a quirk into an attack surface — register the name the models keep inventing, wait for an unchecked install, and you're inside the build. Python Software Foundation developer-in-residence Seth Larson coined the term for it: slopsquatting [C6].
The third mode is the quiet one: logic that is plausible and wrong. CodeRabbit analyzed 470 open-source pull requests and found AI-co-authored PRs averaged 10.83 issues against 6.45 for human-authored ones (roughly 1.7x), with logic and correctness issues 75% more common and readability problems at more than three times the rate [C12]. (CodeRabbit sells AI code review; calibrate accordingly. The direction matches everything else here.)
Even if your reviewers could catch all of this, there aren't enough of them. GitClear (a code-analytics vendor, noted) tracked 211 million changed lines through 2024 and watched copy/pasted lines climb from 8.3% to 12.3% [C7]. Their 2026 update, across 623 million changes, is starker: duplicated blocks up 81% since 2023, copy/paste at 15.7% of changed lines in the first half of 2026, and refactored or moved code — the signature of someone actually maintaining a codebase — down to 3.8% [C8]. Meanwhile, DORA's 2024 report (~3,000 respondents) found a 25% increase in AI adoption was associated with a 1.5% drop in delivery throughput and a 7.2% drop in delivery stability [C9]. More code, less stability, same number of reviewers.
Now the honest counterweight. GitHub's own randomized trial of 202 developers found Copilot-assisted code was 53.2% more likely to pass all ten unit tests on the assigned exercise, with a small readability gain [C15]. GitHub sells Copilot, and this was a single well-specified greenfield task, but take the result seriously anyway. Both findings are true at once: AI code can be functionally solid against a clean spec, and still degrade security and maintainability at repository scale. DORA's 2025 report offers the cleanest synthesis on AI code quality: AI is "a mirror and a multiplier" of what your team already is [C10]. Strong process gets faster. Weak process ships chaos at machine speed. One telling detail from the same report: adoption is at 90%, yet 30% of respondents trust AI-generated code a little or not at all [C10]. Everyone's using it; a third of them don't believe it.
The two reflexive fixes don't hold. "Review harder" loses to arithmetic: the volume numbers above keep climbing, and your senior engineers' attention doesn't. Every hour spent line-reviewing generated boilerplate is an hour not spent on the auth change.
"Have the AI write the tests" fails for a subtler reason: it's circular. As one testing vendor's practitioner guidance puts it, "you cannot use the same AI system to write code and then test it": the same statistical blind spots that shaped the code shape its validation, so the gaps reinforce instead of cancel [C16]. A model that doesn't know your rate limiter should fail closed will happily write a test asserting it fails open. Trust needs an anchor outside the model.
1. Spec-anchored acceptance tests. Test intent, not implementation. Derive cases from your API contracts (OpenAPI or gRPC schemas) and from pre-AI behavior baselines, not from reading the generated diff and guessing what it meant [C16]. This is the counter to plausible-wrong logic: the spec doesn't care how convincing the code looks.
2. Security and dependency scanning in CI. Two non-negotiables. Every new dependency gets verified against the actual registry before install; the 19.7% hallucination rate and the slopsquatting attack path make "it resolved, ship it" untenable [C5][C6]. And every AI-assisted change gets scanned for the OWASP classes models keep flunking, XSS above all [C1][C2]. This is the one AI-generated code security failure mode where the fix is genuinely mechanical.
3. A durable regression suite that lives in your repo. Prompts are disposable; the suite isn't. Every release adds cases, and the asset compounds: next month's confidently-wrong regression gets caught by a test written for last month's feature. If your AI-assisted velocity doubled and your test suite didn't grow, you didn't speed up. You deferred.
4. Human review, reserved for critical paths. Senior eyeballs are now your scarcest resource, so stop spending them uniformly. Auth, payments, data handling, anything touching real user data: human review, every time. The CRUD endpoint behind a feature flag: let the suite and the scanners carry it.
Notice what every move above assumes: someone who knows what the feature was supposed to do. That's the one input no model has — product knowledge, the memory of why the edge case exists. QA for AI-generated code is still a staffing question: whether that person is a full-time hire, an agency, a crowdtest platform, or a subscription engineer is a genuine decision with trade-offs; our buyer's guide to the four ways to buy QA walks through them.
The honest boundary, too: this playbook is overkill for prototypes, internal tools, and genuinely disposable code. The line isn't how the code was written — it's real users plus real data. We've covered how to diagnose that line in a vibe-coded product; the short version is that the moment strangers and their data arrive, "it demos fine" stops being a test plan. (And correctness is only half the audit: the same statelessness that ships plausible-wrong logic also ships design debt — the UX audit is the sibling playbook.)
AI made writing code cheap. It made knowing whether the code is right the expensive part. You don't fix that with a policy memo; you fix it by putting a person on it and judging the results.
That's the shape of Dev on Demand: a QA engineer who works inside your codebase and environment, under your access controls, on one subscription alongside dev and UX engineers. The workflow is AI-augmented (Claude and Copilot as standard), which is exactly the point: engineers who use the tools daily and test what the tools produce. It starts with a 15-minute fit call and a one-task Proof of Quality; tasks run on a 3-day cycle with daily async updates and a task-by-task approval gate, first task within 5 business days. A single stream is $3,495/mo; add or change roles any month, cancel any time, and you own all the IP.
Hand over one real release or one test pass, and judge what comes back. The model won't tell you it's wrong. Someone has to.
How do you test AI-generated code?
Verify it against anchors outside the model: derive acceptance tests from API contracts (OpenAPI or gRPC schemas) and pre-AI behavior baselines [C16], verify every new dependency against the actual registry [C5], scan every AI-assisted change for OWASP vulnerability classes in CI [C1], grow a durable regression suite in your repo, and reserve human review for critical paths like auth, payments, and user data.
How often does AI-generated code have security vulnerabilities?
In Veracode's benchmark of 80 coding tasks across more than 100 LLMs, 45% of generated samples failed security tests against OWASP Top 10 classes [C1][C3]. Java was worst at a 72% failure rate, and cross-site scripting was missed in 86% of applicable samples [C2]. Across 150+ models over two years, security pass rates stayed between 45% and 55% regardless of model generation [C3].
What is slopsquatting?
Slopsquatting, a term coined by Python Software Foundation developer-in-residence Seth Larson, is the attack of registering package names that AI models hallucinate, then waiting for an unchecked install [C6]. A USENIX Security '25 study of 576,000 code samples from 16 LLMs found 19.7% of recommended packages did not exist, and 43% of hallucinated names repeated every time across ten reruns, making the attack predictable [C5][C6].
Can AI write its own tests?
No — it's circular. As one testing vendor's practitioner guidance puts it, "you cannot use the same AI system to write code and then test it": the same statistical blind spots that shaped the code shape its validation, so gaps reinforce instead of cancel [C16]. A model that doesn't know a rate limiter should fail closed will happily write a test asserting it fails open. Tests need an anchor outside the model.
Is AI-generated code worse than human code?
It's different, not uniformly worse. GitHub's randomized trial of 202 developers found Copilot-assisted code was 53.2% more likely to pass all ten unit tests on a well-specified exercise [C15], while CodeRabbit found AI-co-authored PRs averaged 10.83 issues versus 6.45 for human-authored ones [C12]. DORA's 2025 report calls AI "a mirror and a multiplier": strong process gets faster, weak process ships chaos at machine speed [C10].
