AI coding assistants are no longer a tool a few enthusiastic developers experiment with on the side. Tools like GitHub Copilot, Cursor, and Claude Code have become standard equipment on professional engineering teams, and inside many organizations a large and growing share of new code is now drafted by one of them rather than typed from scratch by a person. That adoption has mostly been measured the way any productivity tool gets measured: how much faster did the team ship. Far fewer businesses have asked the harder question, which is what shipped alongside the speed.
The answer researchers keep finding is consistent and specific. Veracode's testing of a large set of language models on security-sensitive coding tasks found that a substantial share of the code they generated failed basic security checks, falling into the same OWASP Top 10 categories — injection flaws, cross-site scripting, broken access control — that have topped that list for years, and the pass rate has not meaningfully improved across successive model generations. That stability is itself the point. A model is trained and evaluated on whether its code runs and does what was asked, not on whether it resists the specific attacks a security team would test for. Nothing about a faster, more fluent model changes what it was actually optimized to get right.
One failure mode shows up constantly enough that it has become the first thing security researchers check for: a secret — an API key, a database password, a signing credential — typed directly into the code because that was the fastest way to get a feature working, with nobody circling back to move it into a secrets manager before the code shipped. An AI assistant asked to "connect to the database" will often produce exactly that shortcut, because it is a common, working pattern in the code it was trained on, and it has no instinct telling it the shortcut is dangerous. Once that credential is committed to a repository, even a private one, it is one misconfigured permission or one compromised laptop away from being exposed.
A second failure mode is less intuitive and harder for a human reviewer to catch by eye. Asked to write code that depends on a library for some capability, a model will occasionally invent a plausible-sounding package name that does not actually exist — a confident guess dressed up as a real dependency. Security researchers have started calling the attack this enables "slopsquatting": publish a real, malicious package under the exact name a model is likely to hallucinate, and wait for a developer, or the same AI assistant on a future run, to install it without checking. The package manager has no way to know the name was invented rather than remembered, and neither, usually, does the person who approved the pull request.
What makes both of these a business problem rather than a developer problem is the gap between how fast adoption moved and how slowly review practices followed it. GitHub has reported Copilot usage well into the tens of millions of developers, with the large majority of Fortune 100 companies using it in some form — this is not a shadow-IT story about one employee's side project, it is mainstream engineering practice inside well-run companies. A code review process built around the mistakes a careful human developer tends to make — a missed edge case, an inconsistent naming convention — was never built to systematically catch a hardcoded secret dropped in because it was convenient, or a dependency that was never real in the first place. Those are not the errors a human author produces often enough to have built a habit of looking for.
The fix is narrower than an overhaul of how the team writes software, and it does not require slowing down adoption of tools that are genuinely making development faster. It requires updating what the review step actually checks for. An automated secret scanner running on every commit, not as an optional linting step but as a hard gate before merge, catches the hardcoded-credential pattern reliably and cheaply. A dependency-verification step that confirms every package a build references actually exists, is maintained, and matches the publisher the team expects closes most of the slopsquatting exposure before it reaches production. And any AI-drafted code touching authentication, payment handling, or customer data deserves a human reviewer's specific attention before merge — passing the test suite is not the same claim as being safe to run against real data.
None of this argues for pulling back from AI coding tools, which are delivering a real and durable productivity gain for teams that would otherwise be stretched thin. It argues for recognizing that the thing writing a meaningful share of a codebase changed, and the process built to catch that code's mistakes did not change with it. The businesses that get burned by this will not be the ones that adopted AI coding assistants. They will be the ones that kept reviewing the output the same way they reviewed a human developer's work, on the assumption that a faster author makes the same kind of mistakes as a slower one.
- ai code generation
- application security
- software supply chain
- vibe coding
- secure development