What a Good Accessibility Score Does Not Tell You

2026-10-13

Somewhere in your accessibility dashboard there is a number, and the number has been improving, and everyone feels good about it. Automated scans have real value: they find missing labels and contrast failures at scale, cheaply, every day. The trouble starts when the team starts treating the score as the thing itself, rather than as a sample of the easy parts of the thing.

An automated checker can only test what is mechanically testable, and most of what makes a store usable or unusable for disabled shoppers is not mechanical. This post maps the gap: what the score sees, what it cannot, and the short list of manual checks that turn a good score into an actually accessible store.

What the machine genuinely sees

Automated tools are excellent at presence and syntax. An image with no alt attribute. A form field with no associated label. Text whose color contrast falls below the ratio. A heading structure that skips levels. ARIA attributes that reference elements that do not exist. These checks catch real barriers, and running them continuously prevents regressions.

Notice what those checks share: they verify that the machinery of accessibility exists, not that it works. The tool confirms the alt attribute is present. It cannot read the alt text and tell you whether image alt equals product photo is a useful description or an empty one. Presence is testable. Quality mostly is not.

What the score cannot see

Tab order is the big one. A scan cannot tell you whether keyboard focus moves through your checkout in the order a human expects, whether focus disappears inside a carousel, or whether your cart drawer traps focus at all. It cannot judge whether your error messages explain how to fix the error, whether your link text makes sense read aloud out of context, or whether the time limit on your checkout is humane.

It also cannot see the journey. Accessibility failures are usually sequence failures: the product page is fine, the cart is fine, but the step between them, a popup, a drawer, a size selector built from clickable divs, strands the shopper. Scans test pages. Shoppers live in flows. The gap between those two views is where the score misleads.

The manual checks that matter

Four checks cover most of the blind spot, and none needs specialist tooling. Complete a purchase with the keyboard alone. Complete one with a screen reader, VoiceOver is already on every Mac and iPhone your team owns. Zoom the store to 200 percent and try to buy something. And run your checkout with animations and motion at the operating system level set to reduce, to see what vestibular disorder users get.

Write down where the journey breaks, in plain language, as tickets. A team that fixes three journey-breaking findings a quarter is doing more for disabled customers than a team that raises its automated score five points, and the findings have the advantage of being undeniably real.

How to report both numbers

Keep the automated score, but demote it. In your reporting it becomes the regression metric: proof that the mechanical baseline is not slipping. Next to it, report journey status: the date the full purchase flow last passed a keyboard pass and a screen reader pass, and the open findings from it. Two numbers, one about syntax and one about humans.

This pairing also protects you legally and ethically. Demand letters and auditors do not care about your dashboard score; they test the journey. A store that can show continuous automated monitoring plus regular manual journey testing has a remediation story. A store with a 100 and a broken checkout has a screenshot.

Frequently asked questions

What does an automated accessibility score actually measure?

The mechanically testable share of accessibility: missing alt attributes, unlabeled fields, color contrast ratios, heading order, and broken ARIA references. Depending on the estimate you use, that is roughly a third of WCAG issues. The rest need human testing to find.

Can a site score 100 and still be inaccessible?

Yes. Alt text that says image, a checkout whose tab order is chaos, a cart drawer that swallows keyboard focus: none of these necessarily lower an automated score, and each can completely block a disabled shopper. The score measures presence of accessibility machinery, not whether the journey works.

What manual accessibility checks should a store run?

Buy something using only the keyboard. Buy something using a screen reader like VoiceOver. Buy something at 200 percent zoom. Each test finds a different class of barrier, and together they cover most of what automated scans miss. Record findings as plain-language tickets.

How often should manual testing happen?

At least quarterly, and after any change to the theme, cart, or checkout. Automated scans should run continuously to catch regressions in the mechanical checks. The manual pass is the audit; the automated score is the smoke alarm between audits.