Keep Your Playwright Suite. Stop Growing It.

A division of labour between code-owned tests and agent coverage

There is a conversation about Playwright that keeps getting framed as a binary: either you write Playwright tests, or you adopt a platform and stop. Both framings are wrong, and the second one is why a lot of engineers stop reading.

Playwright is excellent. It is the best web automation framework available, it has earned its position, and for a specific and important category of tests it is the right answer and will remain the right answer. Nothing in this piece argues otherwise.

The argument is narrower and, I think, harder to disagree with: **not all of your test coverage deserves the same investment.** Some tests warrant hand-written code an engineer owns, reviews, and reasons about. Most of your regression surface does not, and the reason teams end up drowning in Playwright maintenance is that they treat both categories identically.

This is about where to draw that line, and what should absorb the coverage that falls on the other side of it.

Your Playwright suite has an optimal size. It is smaller than where it will end up if you keep adding to it.

What Playwright Actually Solved

Worth being specific, because the credit is deserved and the specifics matter for what follows.

Playwright fixed the framework problem. Auto-waiting eliminated the single largest source of flakiness in Selenium-era suites, the manual synchronization that everyone got wrong occasionally and some people got wrong constantly. One API covers Chromium, Firefox and WebKit, giving genuine Safari coverage rather than a driver approximation. Browser contexts made parallel isolation cheap. Trace viewer made debugging a failed CI run something you can actually do.

The developer experience is genuinely excellent, the documentation is excellent, and Microsoft’s backing means it is not going anywhere. If you are starting a new suite in 2026 and you want code-first testing, Playwright is the correct default and it is not close.

What Playwright did not solve, and never claimed to, is who writes the tests and what happens to them over time.

The Problem Playwright Didn’t Solve: Suite Growth

Every code-owned test carries a fixed recurring cost. Someone wrote it, someone reviews it when it changes, someone repairs it when the application moves underneath it. That cost is roughly constant per test, and it does not decline as the suite grows.

Which produces a predictable trajectory. At fifty tests, maintenance is invisible. At two hundred, it is a recognized part of the sprint. At six hundred, a meaningful proportion of your automation engineers’ week goes to keeping existing tests passing rather than extending coverage, and industry benchmarks put test maintenance at 30–50% of build cost annually test maintenance economics, with Capgemini’s World Quality Report documenting maintenance consuming 30–40% of QA engineering capacity in organizations running active UI automation.

The uncomfortable implication: **success at expanding coverage is what creates the problem.** A team that does its job well generates more maintenance for itself, and the marginal test added at month eighteen costs the same to maintain as the first one, while being, almost by definition, less important than the first one.

None of this is a Playwright failing. It is a property of code-owned tests in general, and it was equally true of Selenium, Cypress and everything before them. Playwright made each individual test more reliable and more pleasant to write. It did not change the fact that each one is an asset with an ongoing carrying cost.

Not All Coverage Deserves the Same Investment

Here is the reframe that makes the rest of this tractable.

Ask what proportion of your Playwright suite consists of tests where the specific implementation genuinely matters, where an engineer needed to reason about the setup, control the sequencing, handle a complex condition, or express something subtle enough that reading the code is how the team understands intended behavior.

For most teams, that is a real and important set. It is also, honestly, a minority of the suite. The rest is breadth: verifying that the settings page still loads, that the export button still exports that the form on the third tab still validates. Necessary coverage, you would notice if it broke, but nothing about it requires an engineer’s judgment to express.

The tests where your engineers’ judgment matters are worth writing in code. The ones where it doesn’t are what’s eating their week.

Treating both categories the same way is the actual mistake, and it is an understandable one, because until recently there was no other option. Every test had to be code because code was the only way to automate anything. That constraint has changed, and suite composition should change with it.

The Division of Labor

A working split, based on what actually determines which side a test belongs on:

The clarifying question for any given test: **if this breaks, do I want an engineer to look at it?** For the checkout flow, yes, emphatically, and quickly. For the fourteenth admin settings screen, the honest answer is that you want it to work and you do not want anyone spending forty minutes on it.

Tests in the second category are not less important than the first. They are less deserving of engineering attention per unit of coverage, which is a different claim and the one that actually matters for how you allocate a team.

How This Works in Practice

The two layers run differently, and keeping them separate is most of the implementation.

Different triggers

Code-owned Playwright tests belong on the fast path: pre-merge, on every pull request, blocking. They are the gate. Because the suite is deliberately capped, it stays fast enough to sit there without anyone negotiating for it, which is itself an argument for capping it.

Agent coverage runs on the broad path: post-merge, nightly, across the full browser and device matrix. It is not gating the merge, it is telling you what the merge did to the other nine-tenths of the application.

Different ownership

Playwright tests live in the repository, are reviewed like code, and are owned by the engineers who wrote them. Agent coverage is defined in plain language and owned by whoever owns the process, frequently a QA analyst or product owner rather than an engineer. That is the point: the ownership model matches who actually understands the flow.

Different failure handling

A failure in the Playwright suite blocks a merge and gets an engineer’s attention immediately, which is appropriate because you capped the suite at things that warrant it. A failure in agent coverage is triaged automatically, classified, and escalated only if it looks like a genuine regression, which is what stops broad coverage from generating triage load proportional to its breadth.

Different growth rates

This is the operational heart of it. The Playwright suite should grow slowly and deliberately, a new test only when something genuinely warrants code ownership. Agent coverage should grow freely, because its marginal cost is close to zero. Over time the ratio shifts heavily toward agents, and total maintenance stays roughly flat instead of scaling with coverage.

What This Is Not

Being precise here, because the coexistence argument only works if it is not a replacement pitch wearing a different jacket.

• This is not a migration. Nothing asks you to convert your Playwright suite. The tests you have that deserve code ownership should stay exactly as they are, in your repository, owned by your engineers.
• Sofy does not emit Playwright code. Agents do not generate TypeScript for your team to review, and they do not import existing Playwright tests. If having reviewable code as the output is a requirement for you, that is a real reason to choose differently, and you should weigh it honestly.
• This is not “agents are better than Playwright.” For the tests that belong in the code-owned layer, hand-written Playwright is better, and an agent is the wrong tool. The argument is about which tests belong in that layer, not about which tool wins.
• This does not eliminate engineering involvement. Someone still decides what to cover, sets the boundary between layers, and handles the failures that turn out to be real. It changes what engineers spend the week on, not whether they are needed.

Where to Draw Your Line

A practical exercise, and one worth doing before evaluating any tool. Take your current Playwright suite and sort every test into one of three buckets:

• Keep in code. If this fails, an engineer should investigate within the hour. The implementation required judgment. Reading it is how someone understands the intended behaviour.
• Move to breadth coverage. Necessary, but if it fails you want it diagnosed automatically and escalated only if real. Nobody would choose to spend an afternoon on it.
• Delete. Most mature suites carry 20–35% dead weight, stale tests, duplicates, coverage of features that no longer exist. Do this pass first regardless of anything else you decide; it is free and it makes the other two buckets honest.

The ratio that falls out is usually informative on its own. Teams commonly find their genuinely code-worthy set is a quarter to a third of what they are maintaining, which quantifies exactly how much engineering time is going to coverage that never required an engineer.

Do the Sort Even If You Buy Nothing The three-bucket exercise is worth an afternoon regardless of tooling decisions. Deleting the dead weight and knowing your true code-worthy ratio makes every subsequent conversation, about headcount, about tools, about coverage targets, a better one.

Where Sofy Fits

Sofy Web testing agents are built for the second layer, and only the second layer.

Coverage is described in plain language rather than written in code, so a QA analyst or product owner can define it without engineering involvement. Agents identify elements contextually rather than through selectors, so ordinary UI churn does not break them and there is no locator maintenance to absorb. When the application changes structurally, agents adapt rather than failing and queuing for repair.

Execution runs across Chrome, Safari, Firefox and Edge, with performance and security signal captured in the same run rather than requiring separate passes. Failures are classified automatically and escalated only when they look like genuine regressions, which is what keeps a large breadth layer from generating triage volume proportional to its size.

What it does not do is replace the judgment layer. Your critical-path tests, the ones your engineers reasoned carefully about, should stay in Playwright where they can be read, reviewed and controlled line by line. Sofy exists so that layer can stay small and stay fast.

Frequently Asked Questions

Should we replace Playwright with AI agents?

No. Playwright is the right tool for tests where the implementation genuinely matters, critical paths, complex setup, cases where reading the code is how the team understands intended behaviour. The useful change is capping that suite at what genuinely deserves code ownership and letting a separate layer absorb the broad regression surface, rather than migrating anything.

Can Sofy import or generate Playwright tests?

No. Sofy agents do not emit TypeScript for review and do not import existing Playwright suites. That is a deliberate architectural difference rather than a roadmap gap, there is no generated artifact because the agent re-derives how to validate on each run. If reviewable code as output is a requirement for your team, that is a legitimate reason to choose a different tool, and worth weighing before a trial.

How big should a Playwright suite be?

Smaller than most teams let theirs become. The useful test is not a number but a question: for each test, if this fails, do you want an engineer to investigate it? Where the honest answer is no, the test is providing coverage without warranting code ownership. Teams doing this sort commonly find their genuinely code-worthy set is a quarter to a third of what they currently maintain.

Doesn’t running two layers add complexity?

It adds one boundary decision and removes a growing maintenance burden. In practice the layers separate cleanly: Playwright runs pre-merge as a fast blocking gate, agent coverage runs post-merge across the full browser matrix. Different triggers, different owners, different failure handling. The complexity of the split is small next to the complexity of a single suite that keeps growing.

What about Playwright tests that keep breaking?

Those are the strongest candidates for the breadth layer, and it is worth asking why they break. If the answer is that the UI churns and the locators follow it, the test is absorbing maintenance without requiring judgment, which is precisely the profile of a test that belongs on the other side of the line.

The Bottom Line

Playwright is not the problem, and replacing it is not the answer. It solved the framework problem convincingly and remains the correct default for code-first web testing.

What it did not solve, what no framework solves, is that every code-owned test carries a permanent carrying cost, and that teams keep adding to the suite as though coverage were free. The result is that the tests deserving your engineers’ attention end up competing for it with hundreds that never did.

Keep the suite. Cap it at the tests that genuinely warrant code ownership, where an engineer’s judgment is doing real work. Let something else carry the breadth. Your Playwright suite gets faster, your engineers get their week back, and coverage stops being limited by how much maintenance your team can absorb.