Tracking Accessibility Error Rates: The Metric That Shows Whether Fixes Stick

2026-10-16

Most accessibility programs report audit totals: how many issues the last audit found. That number is nearly useless for judging progress, because it swings with audit scope, page selection, and tool changes. A bigger audit finds more issues and the program looks worse for having looked harder.

Error rate tracking fixes the denominator problem. By measuring defects per sampled page or per release, weighted by severity, a program can see whether new code is getting cleaner over time, which is the only trend that predicts where the total backlog is heading.

Define the rate before you measure it

Pick a stable sampling method first: the same set of core templates and critical journeys, tested the same way, on a regular cadence. The rate is confirmed barriers per page in that sample. Changing the sample or the method mid year silently rewrites the trend, so document both and hold them steady.

Separate the new release rate from the legacy rate. Barriers introduced in the last quarter of releases, per release, tell you about current development practice. The legacy backlog tells you about history. Programs that blend the two cannot tell whether their training and tooling investments are working.

Weight by severity, honestly

A raw count treats a keyboard trap in checkout as equal to a low contrast footer link. Severity weighting, critical, serious, moderate, minor, with published definitions, makes the rate reflect user impact. A program can then report a weighted rate for executives and a raw count for the teams doing the fixing, and both audiences get the truth in their own units.

Guard the definitions against drift. Severity inflation and deflation are both tempting when a trend line becomes a performance target. Keep worked examples of each severity level, review borderline calls in a group, and the weighting stays a measurement tool instead of a negotiation.

Wire the rate into releases

The most useful error rate is computed at release time: automated checks in the pipeline plus a manual sample of changed screens before launch. That gives a defects per release figure that teams can watch fall as design system fixes, linting rules, and component standards take hold.

When a release spikes, the rate tells you where to look: a new third party widget, a rushed template, a component fork. Without per release measurement, that spike hides inside the next annual audit, months after the cheap moment to fix it has passed.

Report the trend, not the snapshot

Executives should see three lines: new release error rate trending down or not, backlog burn rate on legacy issues, and the severity mix of what remains. Those three answer the only questions that matter: are we building it right now, are we fixing the past, and what risk is left.

Publish the methodology beside the numbers. A rate nobody can interrogate becomes a vanity metric within a year. A rate with a stated sample, severity model, and testing method invites the scrutiny that keeps it honest, which is what a compliance metric is for.

Frequently asked questions

What is an accessibility error rate?

The number of confirmed accessibility barriers per page in a fixed sample, or per software release, tracked over time and usually weighted by severity. It measures whether output quality is improving, which audit totals alone cannot show.

Why are audit issue totals a poor progress metric?

Because they depend on audit scope and method. A wider audit finds more issues and makes a program look worse for looking harder. Rates with stable denominators separate real quality change from measurement change.

Should severity weighting be used?

Yes, with published definitions and examples. A critical barrier in checkout should count more than a minor issue in a footer, and weighting makes reported trends reflect user impact rather than raw counts.

How often should the rate be measured?

On a regular cadence for the standing sample, commonly quarterly, and at release time for changed screens. Release measurement catches regressions while they are still cheap to fix.