Test Automation That Survives Contact With a Real Product

Almost every team we work with has a test suite. Rather fewer have a test suite anybody trusts. The failure mode is consistent: a burst of enthusiasm produces two hundred tests, a handful of them fail intermittently for reasons nobody can reproduce, the team learns to re-run the pipeline until it goes green, and within a few months the suite has become a slow ritual that catches nothing. A suite that is routinely ignored is worse than no suite, because it costs time and provides false confidence.

The root cause is usually the shape of the suite rather than the tooling. Teams write end-to-end tests because they feel the most reassuring — they exercise the real thing — and end-to-end tests are exactly the ones that are slowest, flakiest and hardest to debug. A healthy distribution is weighted the other way: a large base of fast unit and integration tests that run in seconds, a thin layer of end-to-end tests covering the journeys that actually produce revenue, and static analysis catching the whole class of mistakes that never needs a test at all.

The tooling question is largely settled. Playwright has become the default for browser automation because it removed the single biggest source of flakiness: it waits for elements to be actionable rather than requiring you to sprinkle sleeps through the test, and its assertions retry instead of failing on the first read. For React Native, Maestro and Detox occupy the same position. Vitest and Testing Library handle the layer beneath. None of this is controversial any more, which means the interesting decisions are all about what you test and how you keep it honest.

Rules that keep a suite trustworthy :

• Select elements the way a user finds them — by role, label and visible text, not by CSS class or test ID everywhere
• Never assert on a sleep; assert on a condition and let the runner retry
• Each test creates the data it needs and cleans up after itself, so tests can run in any order and in parallel
• Mock the network at the boundary, not inside your own modules — so refactors do not break tests
• One reason to fail per test, so a red result tells you where to look
• A flaky test is a broken test: quarantine it the same day, with an owner and a deadline
• The whole pull request suite finishes in under ten minutes, or people stop waiting for it

Where AI-generated tests help, and where they quietly hurt

Asking an assistant to generate tests for an existing module is genuinely useful for coverage of edge cases a human would skip — empty arrays, unusual dates, boundary values. The trap is that generated tests describe what the code currently does, not what it is supposed to do. Point one at a function containing a rounding bug and you get a test asserting the wrong total, permanently, with a confident name. Generated tests are a first draft that needs a reader who knows the requirement; used as a coverage target they cement whatever behaviour happened to exist on the day they were written.

Building the suite in the order that pays

Coverage percentage deserves a word of warning. It is a useful smell — a module at four per cent is probably untested in the ways that matter — and a terrible target. Teams given a number to hit write tests that execute code without asserting anything meaningful about it, and the number goes up while the risk stays exactly where it was. Better questions: when we last broke production, would this suite have caught it? How long does a developer wait to find out they have broken something?

Our recommendation for a team starting from very little is deliberately modest. Pick the three critical journeys, get them green and reliable, wire them into the pipeline as a required check, and add a test to the suite every time a bug reaches production — written from the bug report, before the fix. Within a quarter that habit produces a suite shaped by your actual failure modes rather than by someone's guess about them, and it stays fast enough that nobody is tempted to skip it.