Why we built on Playwright, and why you can leave us anytime

Early on, one of us asked a question that sounded like a joke and turned out to be a design constraint: "If a customer fires us, what do they keep?"

With a lot of testing tools, the honest answer is "a dashboard login that stops working." The tests live in a proprietary format, run on a proprietary runner, and make sense only inside the product that generated them. Leave, and your regression suite leaves with you.

We didn't want to build that. So every test Testomatica writes is a Playwright test, in TypeScript, that runs with the standard tooling and nothing else.

Why Playwright

We looked at the usual options. Playwright, which Microsoft released in 2020, won for reasons that are mostly boring, which is what you want from a foundation:

• It drives Chromium, Firefox and WebKit from one API, so a "Safari on a phone" persona is a real WebKit engine with a mobile viewport, not a guess.
• Browser contexts are cheap and isolated. Our referee replays every finding in a clean session, and spinning up a fresh context per replay costs very little.
• Network interception is built in. That's how we block write requests on read-only staging without touching the site.
• Its tracing is excellent. A trace file holds actions, DOM snapshots, console output and network activity, and it's the backbone of the evidence we keep on every run.
• Auto-waiting removes a whole category of flaky "element not ready" failures that we'd otherwise have to fight in every test.

None of that is exotic. That's the point. A huge number of developers already know Playwright, and the docs are good.

What a generated test looks like

When a finding is confirmed, the test writer turns it into a spec file. Here's roughly what one for a sort-order bug looks like:

import { test, expect } from '@playwright/test';

test('lowest price sort orders by the price the customer pays', async ({ page }) => {
  await page.goto('/category/office-chairs?sort=price_asc');

  const prices = await page
    .locator('[data-testid="product-card"] [data-testid="final-price"]')
    .allInnerTexts();

  const values = prices.map((p) => Number(p.replace(/[^0-9.]/g, '')));

  for (let i = 1; i < values.length; i++) {
    expect(values[i]).toBeGreaterThanOrEqual(values[i - 1]);
  }
});

No custom DSL, no magic import from our servers, no API key. Just a test a developer could have written, with a name that says what broke.

Run it without us

Put that file in your repo and run:

npx playwright test

It works. Testomatica doesn't need to be running, reachable or paid for. The standard reporters work too, so you can use list in a terminal, html for a browsable report, or junit for your CI dashboard. Turn on tracing in your config and you get the same trace files we use, viewable with npx playwright show-trace.

What Testomatica adds on top is the stuff around the test: deciding which tests to run for a given diff, keeping new tests in quarantine until they've proven stable, retrying sensibly, separating infrastructure failures from real ones, and feeding results into the release gate. That's real value, and we think it's worth paying for. But it's orchestration. The tests themselves are yours.

No lock-in, on purpose

We'll admit there's a business argument against this. Making it easy to leave makes it easy to leave.

We think the opposite argument is stronger. If you know you keep every test, trying us is a smaller bet. If the tests are plain code, your developers can review them, fix them and extend them, which makes them better. And if we ever stop being useful to you, you walk away with a regression suite instead of a hole.

We're getting ready to point the full pipeline at a real staging site soon, and the first tests it produces will be the real proof. If they don't read like something a person on your team would happily maintain, we'll have more work to do.