Before the study started, the developers forecast that AI would make them 24% faster. After they finished, they estimated it had made them 20% faster. The stopwatch said they were 19% slower [C1][C2].
That gap between felt speed and measured speed is the defining fact about AI augmented developers right now. The rest of the evidence sorts into a clear pattern once you stop averaging it.
Key takeaways
Does AI make developers faster? Sometimes — the task decides. In METR's randomized trial, experienced developers on mature codebases were 19% slower with AI while feeling 20% faster [C2]. On well-specified greenfield work, developers finished 55.8% faster [C6]. There is no single number.
Those three numbers come from one experiment: METR's randomized controlled trial of 16 experienced open-source developers working 246 real tasks in their own repositories, using early-2025 tooling: Cursor Pro with Claude 3.5 and 3.7 [C1]. Same people, same tasks, same tools. Forecast: 24% faster. Felt: 20% faster. Measured: 19% slower [C2].
Read that as an anti-AI result and you've misread it. It's a calibration result. The developers weren't lying and they weren't lazy. They were wrong about their own speed, in the direction the marketing primed them to be. Felt velocity is no longer a measurement.
This matters because the rollout is done. DORA's 2025 report puts AI adoption among software professionals at 90%, up 14% year over year [C9]. Sundar Pichai says "more than a quarter of all new code at Google is generated by AI, then reviewed and accepted by engineers" [C15] — and notice the second clause doing quiet work in that sentence. Even at Google, the number comes with a review gate attached.
We ship with Claude and Copilot every working day, so this is not a takedown. It's a map of when the tools pay you and when they bill you.
On greenfield, well-specified work, yes: controlled studies show large speedups [C5][C6]. For experienced developers inside mature codebases they know deeply, the only randomized trials to date show a slowdown [C1][C4]. There is no single answer; the task type decides.
METR's own caveat is right there in the writeup: "We do not claim that our developers or repositories represent a majority or plurality of software development work" [C3]. The setting was deliberately hard for AI: mature repositories averaging 22,000+ stars and over a million lines of code, worked on by developers with roughly five years of familiarity with that specific codebase [C1][C3]. This is expert-on-home-turf territory, where most of the job is knowing where things are and what will break.
METR re-ran the setup, publishing an update in February 2026. Central estimate for returning developers: still 18% slower, with a confidence interval running from −38% to +9%. New recruits landed at −4%, CI −15% to +9% [C4]. METR's own characterization: the raw results "show some evidence for speedup" but "only very weak evidence" once you account for selection effects [C4]. The honest read is three-part: the gap is narrowing, there is still no confirmed RCT speedup for experienced developers on mature code, and developers are still self-reporting large speedups — the kind of estimate METR's first study showed to be unreliable.
Now hold that next to the studies the vendors quote. GitHub's own RCT of 202 developers with five-plus years of experience found Copilot-assisted code had a "53.2% greater likelihood of passing all 10 unit tests," with a 3.62% readability edge [C5]. Real result, on greenfield API endpoints, and it's GitHub studying GitHub, which they'd be the first to note you should weigh. The famous "55% faster" figure traces back further: Peng et al. in 2023 found developers "completed the task 55.8% faster" — and the task was "implement an HTTP server in JavaScript as quickly as possible" [C6]. GitHub's marketing version is the same task family, 95 developers, published in 2022 [C7].
None of these studies contradicts the others. They measured different work.
Line the results up by task type and the pattern is hard to miss. The biggest gains show up on well-specified, from-scratch, boilerplate-heavy work: an HTTP server against a spec [C6], fresh API endpoints [C5]. The smallest — or negative — results show up where experienced people work inside codebases they know deeply, where the constraint is context, not keystrokes [C1][C3].
Peng's paper says the quiet part out loud: the heterogeneity in its results "show[s] promise for AI pair programmers to help people transition into software development careers" [C6]. Bigger gains for less-experienced developers on routine work. That's the paper's own framing, and it's the honest version of the pitch you're currently hearing, which goes "AI makes your mid-levels ship like seniors."
It doesn't. It compresses the routine parts of the job, and routine work is a larger share of a mid-level's week; that's the whole effect. What it does not add is the judgment that makes a senior a senior: knowing which change is safe, which abstraction will rot, which test actually protects you. AI adds throughput to whatever judgment already exists. Multiply good judgment and you get velocity. Multiply thin judgment and you get more code, faster, which is not the same thing.
Individual studies split by task type. Organization-level data adds the part no lab study captures: what happens downstream of the keyboard.
DORA's 2024 report, across roughly 3,000 respondents, found that a 25% increase in AI adoption was associated with a 1.5% drop in delivery throughput and a 7.2% drop in delivery stability, numbers RedMonk pulled out in its analysis [C8]. RedMonk's Rachel Stephens named the bottleneck directly: writing code was never the bottleneck; putting good code into production is [C8]. Speed up a non-constraint and the constraint just gets a bigger queue.
The code-quality data corroborates the queue. GitClear — a code-analytics vendor, so weigh accordingly — measured duplicated code rising from 40.3 to 73.0 on its index, an 81% increase over 2023; moved code (the fingerprint of refactoring) collapsing from 13% of changes to 3.8%; copy-pasted code climbing from 9.4% in 2022 to 15.7% in the first half of 2026. Their summary: "the structural habits that keep codebases maintainable have eroded across every signal we measure" [C14].
And developers themselves are recalibrating in real time. Stack Overflow's 2025 survey has 84% using or planning to use AI tools, up from 76%, with 51% of professional developers using them daily. In the same survey, favorability fell from above 70% to 60%, and 46% now distrust AI output accuracy against 33% who trust it (3.1% trust it highly) [C13]. A slim majority, 52%, report a positive productivity effect [C13] — any positive effect, note, not a transformative one. Everyone is using it. Fewer believe it.
DORA's 2025 report (the 90%-adoption one [C9]) resolves the contradiction by refusing to average it. Their framing: AI is "a mirror and a multiplier" of the organization it lands in [C11]. Bluntly: "AI is an amplifier. It magnifies the strengths of high-performing organizations and the dysfunctions of struggling ones" [C12]. The trust numbers match the mirror: only 24% of practitioners trust AI output a great deal or a lot, while 30% trust it a little or not at all [C10] — the same tool producing opposite experiences in different systems.
DORA goes further than diagnosis and names the seven capabilities that separate the amplified from the exposed [C12]:
Three of those a CTO controls this quarter without a budget line: small batches, version control discipline, and internal platform quality. On small batches, DORA is specific about why it's now load-bearing: "AI can easily generate massive blocks of code, which are hard to review and test. Enforcing the discipline of small batches counteracts this risk…" [C12]. A 2,000-line AI pull request isn't reviewed; it's skimmed and blessed. Tests are the other half of that safety net. We've written a separate playbook on QA for AI-generated code that pairs with this.
Disclosure where it's due: DORA is Google-owned and Google sells AI development tooling. But the capabilities list is process advice, not product advice, and it agrees with the independent evidence above — including the evidence that's inconvenient for AI vendors. The playbook is boring on purpose. The organizations winning with AI are doing more of the boring stuff, not less.
Every staffing pitch you receive now says "AI-augmented engineers." The adjective is free. The loop around it is the product, and the loop is what you should interrogate, with any vendor, including us. If you're evaluating outside capacity because you can't hire engineers fast enough, the same four questions apply to every model:
What's the batch size, and what enforces it? Where does the client approve or reject work, and how often? Who writes the tests, and against what? And how is output measured: shipped, accepted work, or something gameable like PR count?
DevOD's answers, concretely: Claude and Copilot are standard in the workflow, and every piece of work runs through a task-by-task approval gate on a 3-day task cycle with daily async updates. That structure isn't an AI policy memo — it's small batches enforced by the delivery model itself. A task that can't be scoped, built, and reviewed in three days gets split until it can. You see every batch before the next one starts.
Apply this post's own finding to your buying process. The METR developers felt 20% faster while running 19% slow [C2], so don't accept felt velocity as evidence, whether it's your team's or a vendor's. Time real work.
That's the shape of our front door on purpose: a 15-minute fit call, then a one-task Proof of Quality before any subscription: a real task from your backlog, so you judge shipped output rather than a pitch deck. If it holds up, a single Dev, QA, or UX stream is $3,495/mo, and you can cancel any time.
The tool is real. The multiplier is yours.
Does AI make developers faster?
It depends on the work. Experienced developers in mature codebases they knew deeply measured 19% slower with AI in METR's randomized trial [C2], while developers on well-specified greenfield tasks completed work 55.8% faster [C6]. The task type — not the tool — decides the outcome.
What did the METR study find?
METR ran a randomized controlled trial of 16 experienced open-source developers across 246 real tasks in their own repositories, using Cursor Pro with Claude 3.5 and 3.7 [C1]. Developers forecast a 24% speedup and felt 20% faster afterward; measured, they were 19% slower [C2]. A February 2026 re-run with late-2025 tools still put returning developers at 18% slower as the central estimate [C4].
Why do developers think AI makes them faster when it doesn't?
It's a calibration failure, not laziness. The METR developers misjudged their own speed in the direction the marketing primed them to [C2], and in the 2026 re-run developers were still self-reporting speedups the measurements didn't confirm [C4]. Felt velocity is not a measurement; time real work instead.
What are the DORA AI capabilities?
DORA names seven capabilities that separate organizations AI amplifies from organizations it exposes: a clear and communicated AI stance, healthy data ecosystems, AI-accessible internal data, strong version control practices, working in small batches, user-centric focus, and quality internal platforms [C12]. A CTO controls three of those this quarter without a budget line: small batches, version control discipline, and platform quality.
Is AI-generated code lower quality?
The signals are mixed. GitHub's RCT found Copilot-assisted code 53.2% more likely to pass all ten unit tests on greenfield endpoints [C5]. But GitClear measured duplicated code up 81% over 2023 with refactoring collapsing from 13% of changes to 3.8% [C14], and DORA 2024 tied a 25% rise in AI adoption to a 7.2% drop in delivery stability [C8].
