The Bugs AI Code Leaves for Someone Else to Find

People keep asking some version of the same question: does testing still matter once AI is writing most of the code? There are two easy answers, and both are kind of lazy. One is “of course it does, it always has,” which doesn’t actually explain anything. The other is that AI output is so different from human code that it needs some whole new philosophy of validation. Neither answer looks at what’s actually happening to the code once it ships, which turns out to be the only part that matters.

A fair number of developers now lean on AI for most of their day, and most of them say it’s genuinely helping, faster output, fewer slow afternoons stuck on boilerplate. That part isn’t in dispute, and it’s right there in Google’s own survey of developers on how they’re actually using these tools. What’s less talked about, and sitting in that same survey, is how many of those same developers, when asked directly, say they don’t fully trust what the tool just handed them. That’s an odd place to land: shipping more AI-touched code than ever while still eyeing a good chunk of it sideways. That gap, between what’s going out the door and what anyone actually believes about it, is exactly the gap QA was always meant to cover, and right now it’s wider than it’s been in a while.

A few independent studies looked at what’s actually inside that gap, comparing AI-written code against human-written code on the same kinds of projects, and the pattern across them is consistent even though they come from different companies measuring different things. One comparison of real open source pull requests found AI-generated code carrying noticeably more defects than human-written code on the same kind of task, split pretty evenly across plain logic mistakes, maintainability problems, and security issues. Separately, a team that tracks hundreds of millions of real code changes over time found duplicated code climbing and genuine refactoring work falling off a cliff, meaning the same logic gets copy pasted around instead of consolidated, and fewer people go back to clean up what’s already there. None of that shows up as a crash on day one. It shows up over a year later as a codebase nobody wants to touch, which is a slower and more expensive kind of failure than a bug ticket.

Security follows the same shape. A security-focused study that ran generated code from a wide range of models through real vulnerability checks found that the rate at which generated code actually passes has barely moved in the last year, even while the models themselves have gotten noticeably better at everything else. The code compiles, it runs, it looks right. Whether it’s actually safe is a separate question that capability improvements haven’t been answering. Getting better at writing code and getting better at writing secure code turned out to be two different skills improving on two different timelines, and only one of them is the one that matters once the thing is live.

Put all of that together and the real story here is about where the risk moved, not about whether AI is good or bad at writing code. A clean test run and code that compiles were never a perfect stand-in for “this is correct,” but they used to be good enough, back when the main source of bugs was a person making an ordinary human mistake. AI shifts the failure mode toward exactly the stuff a quick skim and a green test suite won’t catch: logic quietly duplicated in three places instead of one, an error handler that swallows a problem instead of surfacing it, a security mistake that’s common enough to have its own name and its own well-known fix. None of that trips an existing test. All of it is exactly what a real second set of eyes, someone actually reading the code rather than trusting that it ran, is for. So no, the answer isn’t that testing needs reinventing as some grand new discipline for the age of AI. It’s much plainer than that. The thing QA was always supposed to catch, code that looks fine and isn’t, just got a lot more of it to catch.