What automated accessibility scans can't see

Today is Global Accessibility Awareness Day, and a lot of teams will mark it the same way: someone runs an accessibility scanner on the homepage, gets a list of issues, and fixes a few contrast problems before lunch.

That's genuinely worth doing. It's also where a lot of teams stop, and it's worth being clear about why that isn't enough.

What scanners are good at

We're planning on axe-core, the open-source engine from Deque, as the accessibility layer in our quality checks. The idea is to run it on every page a test or an agent visits. It's very good at a specific kind of problem: things you can decide by looking at the page's structure.

• An image with no alt attribute.
• A form input with no associated label.
• Text whose color contrast against its background is below the WCAG threshold.
• ARIA attributes that are misspelled, invalid, or used on an element that doesn't support them.
• A page with no lang attribute, or a button with no accessible name at all.

These are real barriers, and they're mechanical to detect. A rule either finds a label or it doesn't. That makes them a great fit for the deterministic side of our design: they run without a model, they give the same answer every time, and they can be part of a gate.

WCAG 2.2, published in 2023, added several success criteria, and some of them are partly checkable this way too. Target size is one: on a phone, a 12-pixel "remove item" icon next to a 12-pixel "add to wishlist" icon is a problem a tool can measure.

What they can't see

Here's where it gets harder. A scanner can tell you an image has alt text. It can't tell you the alt text is useless.

alt="image" passes. alt="IMG_4471.jpg" passes. alt="Product photo" on all forty products in a category passes. A screen reader user browsing that category hears "Product photo, Product photo, Product photo" and learns nothing. Whether a description is meaningful is a judgment, and the rule engine isn't the place for judgment.

The same goes for most of what makes a page usable without a mouse or without sight:

Keyboard flow. Can you tab from the search box to the results, into a product, pick a size, add it to the cart and check out, without a mouse? Does focus go somewhere sensible after you close the size guide modal, or does it jump back to the top of the page? Is there a focus trap in the cookie banner that you can't tab out of? A scanner looks at one page state. Keyboard flow is a journey.

What the screen reader actually says. Markup can be technically valid and still produce nonsense when read aloud. A price announced as "dollar nineteen ninety-nine strikethrough dollar twenty-four ninety-nine" with no hint of which one you'll pay. A "Loading" message that's never announced, so the user doesn't know anything is happening. A cart update that changes the total silently. These are only visible when someone listens.

Focus you can see. WCAG 2.2 also added that a focused element shouldn't be entirely hidden by other content, like a sticky header covering the link you just tabbed to. Some of that can be approximated in code. Whether it actually feels usable is something you find out by trying.

Where agents fit, and where they don't

We're designing a keyboard-only persona for our explorer agents: a browser configuration with no mouse events, moving through a journey with Tab, Enter and the arrow keys. It can catch things a static scan misses, like a step in checkout that's simply unreachable from the keyboard. When it gets stuck, that's a finding we can turn into a runnable check.

But we want to be honest about the limit. An agent can tell you focus didn't move. It can't tell you what the experience is like for someone who navigates by screen reader every day. Full validation requires manual testing with assistive technologies, done by people who know how those tools are actually used, and ideally including people who rely on them. No scanner, and no agent, replaces that.

So on our coverage map, accessibility gets split across all three executors. Code will run the axe-core rules on every page. Agents walk keyboard journeys and flag where they get stuck. A human, with a screen reader and a keyboard, owns the final call.

If you're running a scan today, good. Then try this: unplug your mouse, open your own checkout, and see how far you get.