SysAdmin Day: thank you for keeping staging alive

Today is System Administrator Appreciation Day, the last Friday in July. If there's someone on your team who knows why the staging database has that one weird cron job, now is a good time to buy them a coffee.

We've been thinking about sysadmins a lot lately, because nearly everything we're building depends on an environment someone else keeps running.

Staging is a strange place. It's supposed to look like production but nobody treats it like production. Developers deploy to it six times before lunch. QA fills it with test orders named "asdf." Someone runs a load test without telling anyone. A certificate expires on a Sunday. And through all of it, somebody makes sure that on Monday morning it's still there, still reachable, still roughly resembling the real thing.

That person rarely gets thanked when it works. They mostly hear about it when it doesn't.

What we promise staging

Since we're about to become one more thing hitting your staging environment, it seems fair to say how we plan to behave.

We'll only read. Staging is read-only for Testomatica. Requests that could write data, like placing an order, editing a profile, or submitting a form that changes something, are blocked in the browser before they leave. Our agents can look at the checkout. They can't create a thousand orders in it.

We'll check before we start. Every run begins with a preflight: is the site up, does access work, is there budget. If staging is down for maintenance, we notice and wait, instead of hammering it with requests that will all fail.

We won't blame you for outages. If the VPN drops or the site is unreachable, that's recorded as an infrastructure problem, not a product bug. No developer gets a ticket saying checkout is broken because a host was rebooting. If an outage lasts a long time, one clearly labeled infra ticket goes to whoever owns the environment. One ticket, not forty. We wrote more about this in An infrastructure failure is not a test failure.

We'll keep a log. Every action we take is recorded in an audit log, so when someone asks "what was hitting staging at 3:14 a.m.?" there's an answer.

The quiet part of quality

There's a habit in software of crediting quality to whoever wrote the test that caught the bug. Fair enough. But that test only ran because a server was up, a DNS record pointed the right way, a disk hadn't filled with logs, and a VPN stayed connected through the night.

Tests are only as dependable as the ground they stand on, and someone keeps that ground level.

So to the sysadmins, ops folks, platform engineers, and everyone else who answers when staging falls over: thank you. We'll try to be a polite guest.