Manual vs Automated Accessibility Testing: What Each Catches That the Other Misses

The third that scanners see

Automated accessibility scanners are good at what they do, and what they do is narrow. A scanner checks code against rules: missing alt text, insufficient color contrast ratios, form inputs without labels, heading levels that skip. These are real barriers and scanners find them fast, across thousands of pages, for almost no cost. Every accessibility program should run them.

But the commonly cited figure holds up in practice: automated tools detect roughly a third of WCAG failures. The other two thirds require a human. This is not a flaw in the scanners. It is the nature of the problem. Accessibility is about whether a person can use the thing, and code inspection can only approximate that question.

What only manual testing catches

Start with keyboard operability in the real sense. A scanner can verify that interactive elements are focusable. It cannot tell you that the focus order jumps illogically across the page, that a modal traps focus without an escape, or that a custom dropdown requires a mouse. These are the barriers that actually block keyboard users, and they are invisible to automated checks.

Screen reader testing finds the next layer. A scanner confirms that images have alt attributes. It cannot tell you the alt text says image1234.jpg, or that a data table reads as gibberish because the headers are wrong, or that a live region announces every keystroke. Meaning, context, and verbosity are human judgments.

Then there is cognitive and usability testing: instructions that make sense to the author but confuse everyone else, error messages that blame the user, timeouts that punish slow readers. No scanner evaluates whether a human can understand the page. Manual testing with real users, including users with disabilities, is the only method that answers that question.

What only automated testing catches

The comparison runs both ways. Manual testing is slow and sampling-based: a tester covers key flows, not all ten thousand product pages. Scanners cover everything, every deploy, in minutes. Regressions, the alt text that disappeared in last week's template change, the contrast failure in the new banner component, are caught by automation long before a human would notice.

Automation also enforces consistency. Human testers vary in thoroughness and interpretation. A scanner applies the same rules the same way every time, which makes it the right tool for gating deploys and tracking trends. Manual testing tells you whether the site is accessible. Automated testing tells you whether it stayed accessible.

How to budget for both

The practical model is layered. Run automated scans continuously: on every deploy, across the full site, with findings triaged by severity. This is cheap and catches regressions. Schedule manual audits quarterly, or after major redesigns, covering key user journeys with keyboard-only and screen reader testing. This is expensive and catches the barriers that actually generate complaints and lawsuits.

Add user testing annually, or when launching new interaction patterns. Nothing replaces watching a screen reader user attempt your checkout. The findings are fewer than a scan produces but each one matters more, because they describe real blocked users rather than theoretical violations.

Report the methods separately. An accessibility statement that cites only automated scan results is claiming more than it measured. Leadership should see both numbers: the scan pass rate and the manual audit findings. The gap between them is the honest measure of where the program stands.