Somewhere right now a founder is asking "should we do manual or automated testing?" as if it were one decision. It's two questions: what do you automate, and who holds the judgment? Get the split right and the framework choice barely matters. Get it wrong and the world's best Playwright suite will green-light a release your users hate.
Key takeaways
Manual vs automated testing isn't a binary. Automation owns regression: scripted checks of the critical paths that must keep working, run in CI on every commit. Humans own discovery: exploratory testing that finds what no script was written to catch. Pick Playwright, point it at regression, keep judgment human.
Let's clear the most-argued, least-important decision first. In the last week of July 2026, the playwright npm package pulled roughly 68.5 million weekly downloads against Cypress's 7.7 million and selenium-webdriver's 2.0 million [C1]. Now the caveats, in the same breath: npm download counts include every CI reinstall, the playwright package also gets installed for scraping and agent automation rather than testing, and Selenium's Java, Python, and C# distributions never touch npm at all, so that 2.0 million badly understates Selenium's real footprint [C1]. Directional numbers, not precise ones.
But the direction is unmistakable, and it isn't new. Playwright passed Cypress in npm downloads by mid-June 2024, per a crossover analysis from Checkly, who, disclosure, sell Playwright-based monitoring, so they had reason to notice [C2]. Workplace usage tells the same story from a different angle: the State of JS 2024 survey (11,730 respondents) found 3,674 using Playwright at work versus 3,603 for Cypress and 1,130 for Selenium. That's a self-selected sample of JavaScript developers, which again understates Selenium's enterprise presence [C3].
So: greenfield project, pick Playwright, move on. Spend your argument budget on the question that actually determines your quality: what you point the automation at.
Strip the marketing and an automated test is a machine that verifies exactly what it was told to verify. As one testing vendor's commentary puts it, automation checks only what it is programmed to check [C14]. That sounds like a limitation, and it is. It's also a precise job description.
It makes automation perfect for exactly one category: regression. The login flow, the checkout, the API contract, the paths that must keep working release after release after release. A human re-checking those paths does the same work every sprint and produces nothing durable. A scripted check is written once and then runs near-free in CI, forever. Every release adds checks; every check keeps paying. The pass/fail on any given run is a moment — the suite is the asset, and it compounds.
That's the whole case for automation, and it's a strong one. It's just not a case for automating everything.
Two real bills come with that compounding asset.
The first is people. Engineers who write good automation cost more than engineers who execute manual test plans: one QA outsourcing vendor, QA Madness, publishes rate cards showing automation QA rates running 20–50% above manual QA rates in every region they price [C4]. That's a vendor's own rate card, not an industry consensus figure, but the direction matches what anyone hiring in this market has seen.
The second bill is maintenance, and its ugliest line item is flakiness: tests that fail sometimes for reasons that have nothing to do with the code. Google reported back in 2016 that about 16% of its tests exhibited some flakiness, with roughly 1.5% of test executions producing a flaky result. Those figures circulate today via a Playwright-analytics vendor's aggregation, so treat them as secondhand [C10]. More recent and more alarming: Bitrise's 2025 data, as they report it, shows the share of teams experiencing test flakiness climbing from 10% in 2022 to 26% in 2025 [C11]. And a 2019 academic survey by Eck and colleagues found 91% of developers deal with flaky tests at least a few times a year [C12]. Every flaky test is a small tax: someone triages it, reruns it, and in the worst case learns to ignore red builds.
Here's the thing, though: none of this is an argument against automation. It's an argument against automating the wrong things. The trap is paying automation-grade rates to script edge cases and cosmetic checks that will flake, drift, and demand maintenance forever, while checking almost nothing that matters [C14]. Point the machine at regression on critical paths. Be stingy about everything else.
Exploratory testing is not "manual testing," in the sense of a human robotically executing steps a script could run. It's a discovery activity where automation is a checking activity: a skilled tester probing a feature with intent, noticing what's off, following the smell. It surfaces the bug no script was written to find, because nobody knew to look for it [C13].
That category is bigger than it sounds. The new feature nobody has scripted yet. The workflow that technically passes every assertion but makes no sense to an actual user. The form that accepts the input and quietly mangles it downstream. The bug that lives in the assumption, not the code: the spec was wrong, so the tests faithfully verify the wrong behavior. A checking machine can't catch any of that, definitionally. It checks only what it was programmed to check [C14].
The Ministry of Testing community position compresses this to one line: great automation does not absolve you from manual and exploratory testing [C13]. And note that this is judgment work, which matters more, not less, as more of your code is AI-generated. Plausible-wrong logic that compiles, passes its own tests, and does the wrong thing is exactly the failure mode exploratory judgment exists to catch.
"Google and Meta killed QA, so should we" is the most misread precedent in software. Big Tech didn't go 100% automation. They moved testing into the developer role, backed by serious automation infrastructure, and the QA-ish roles that survive are, in The Pragmatic Engineer's summary, "about automation, empowering engineering teams to do testing, or both" [C5]. Testing didn't disappear; ownership moved, and a platform caught what fell.
Copy that if, and only if, you have the infrastructure and scale to catch what falls. The cautionary brackets around the model are instructive. When Indeed eliminated QA roles, one developer there reported that "the overall quality of tests has nosedived" [C6]. A single anecdote, but a pointed one. Meanwhile, companies shipping hardware or operating in regulated domains still run dedicated QA, including exploratory testing, because the cost of a miss is not a hotfix [C7]. And at the other end of the scale, the startup reality is candid: "We are too small to have a dedicated QA person. One of our co-founders does most of the QA…" [C8].
Most fast-shipping small teams live between those brackets: too small for a test platform team, too exposed for co-founder-does-QA-at-midnight. What they need is the split (automation owning regression, human judgment owning discovery), not a shrunken photocopy of Google's org chart. Whether you've hit the point of needing a dedicated person is a separate call; we've mapped when to hire a QA engineer.
The newest version of "we don't need QA" is "the AI writes the tests." And the first half is genuinely true: AI test generation has cut the cost of writing and healing scripts, and that's real progress. The scripting bottleneck that made automation expensive is shrinking.
The caveat comes from an unexpected direction. Skyramp, a company that sells AI testing tooling (so the conflict of interest cuts toward credibility here), put it bluntly in March 2026: "you cannot use the same AI system to write code and then test it" [C9]. Same model, same blind spots. An AI that misunderstands your requirements will encode that misunderstanding in the code and then write tests that lovingly verify it. The tests pass. The feature is wrong.
Testing has always depended on an independent oracle: something outside the code that knows what correct looks like. AI compresses the scripting. The oracle in the loop is still a human with product judgment.
Automate regression: the stable, critical paths that must survive every release, running in CI on every commit. Keep discovery manual: exploratory testing of new features, usability, and release sign-off. Whatever the exploratory pass finds and you fix gets promoted into the automated suite.
That's the whole test automation strategy, small enough to fit on a sticky note:
Who runs this? Notice the split doesn't need two people. It needs one person who can do both: write and maintain the Playwright suite, and run the exploratory pass with real product judgment. That's the shape of a QA engineer on Dev on Demand: an engineer working inside your codebase and environment, on one subscription alongside Dev and UX roles, with an AI-augmented workflow (Claude and Copilot as standard), at $3,495/mo for a single role. Add or change roles any month, cancel any time.
If you want to test the split before you commit to it, there's a cheap experiment: book a 15-minute fit call and hand one real release to a QA engineer as a one-task Proof of Quality: first task ships within 5 business days, tasks run on a 3-day cycle, and you approve each one before the next. Then judge the output on both halves: what the scripts caught, and what only a human saw.
Because that's the whole point. The scripts check what you knew to check. Someone still has to find what you didn't.
No. Rote script-execution work is disappearing into automation, but exploratory testing is a discovery activity automation can't perform: it finds the bugs nobody wrote a script for [C13]. Even Big Tech didn't eliminate testing judgment; it moved that judgment into the developer role, backed by platform infrastructure [C5].
For a greenfield project, Playwright. It pulled roughly 68.5 million weekly npm downloads in late July 2026 against Cypress's 7.7 million (directional numbers: CI reinstalls and non-testing installs inflate them) [C1], and the State of JS 2024 survey already showed slightly more respondents using Playwright at work than Cypress [C3].
Regression on your critical paths: login, checkout, API contracts, the flows that must keep working release after release. Those checks are written once and run near-free in CI, so the suite compounds. Edge cases and cosmetic UI checks are the classic trap; they flake and demand maintenance while covering little that matters [C14].
AI has genuinely cut the cost of writing and maintaining test scripts, but it can't be its own oracle. As Skyramp, itself an AI testing vendor, puts it: "you cannot use the same AI system to write code and then test it" [C9]. The same blind spots recur in generation and validation, so a human with product judgment stays in the loop.
Flaky tests fail intermittently for reasons unrelated to the code. Google reported in 2016 that about 16% of its tests showed some flakiness (figures circulating via secondary aggregation) [C10]; Bitrise's 2025 data, as they report it, shows teams experiencing flakiness rising from 10% in 2022 to 26% in 2025 [C11]; and a 2019 survey found 91% of developers hit them at least a few times a year [C12]. They erode trust in red builds, which is the one thing a suite exists to provide.
