Tabpath field measurement · 31 August 2026 · second edition

The Overlay Gap

Fifty storefronts pay for an accessibility overlay. We confirmed each widget was running, gave it fifteen seconds, and measured what was left.

Sample 313 storefronts → 294 valid → 50 with a confirmed-live overlay Engine axe-core 4.13 · Chromium · 1366×900 Settle 2,500 ms vs 15,000 ms

0/50 storefronts reached zero machine-detectable WCAG failures

1,819 failing element instances still present after the 15-second settle

+143 net change from giving every overlay 6× longer to run

190 of those failures sit inside the overlay widget's own injected markup

An accessibility overlay is a script a store adds to its site, typically for $490 to $3,500 a year, that promises WCAG conformance without touching the underlying code. This is a measurement of what a visitor actually receives.

The first edition of this study covered ten stores. This one covers fifty, drawn from a census of 313 storefronts on the public client rosters of fifteen e-commerce agencies. Scaling the sample corrected one of our own headline numbers and confirmed the other.

Two methodological problems have to be solved before any of this means anything, and most published claims about overlays solve neither.

Problem one: auditing too early

Overlay vendors inject their remediation asynchronously, after their script boots. Audit two seconds after load and you have measured the page before the product you are paying for has run — which proves nothing except impatience. So every page here was measured twice, at 2.5 seconds and at 15 seconds.

Problem two: counting scripts instead of products

A script tag is not a running product. It can be a lapsed subscription, a conditional load, or a leftover include. So each site was checked for the vendor's rendered DOM — [data-acsb], #userwayAccessibilityIcon, [id^="ae_"], #INDmenu-btn — before any claim was made about it.

That check removed seven sites. See below; it is a finding in itself.

What fifteen seconds bought

Fifty storefronts with a confirmed-rendering overlay widget, measured at both settle windows. “Clean” counts stores reaching zero machine-detectable failures.

Vendor	Stores	At 2.5 s	At 15 s	Δ	Critical	Serious	No skip link
accessiBe	31	1,003	1,038	+35	34	75	5
UserWay	14	420	422	+2	14	32	6
AudioEye	3	151	150	−1	2	6	0
EqualWeb	1	57	51	−6	0	2	1
accessiBe + UserWay	1	45	158	+113	2	2	0
Total	50	1,676	1,819	+143	52	117	12

7stores improved at all

30stores did not change by a single instance

13stores got worse as late content loaded

−6the largest improvement recorded, on any store, by any vendor

That last figure is the one to sit with. Across fifty storefronts and four vendors, the best result anyone achieved in fifteen seconds was removing six failing instances from a page that still had fifty-one. Thirty stores were bit-for-bit identical. Critical-severity failures moved from 53 to 52; serious from 119 to 117.

The +113 regression came from one store running both accessiBe and UserWay, where lazy-loaded content arrived during the longer window. That is content appearing late, not overlays causing damage — but it is worth noting that two overlays did not help it either.

Seven stores were paying for a widget that never appeared

Of 57 storefronts whose pages referenced an overlay vendor's script, seven rendered no widget at all after fifteen seconds — no trigger, no button, no vendor DOM. Whether those are lapsed subscriptions, misconfigurations, or conditional loads, we cannot say from outside.

We excluded all seven from every efficacy figure on this page, because it would be dishonest to blame a product for not working when the product was not running. But it carries a warning for anyone else measuring this: counting script tags overstates overlay prevalence by about 12%. Prevalence across our whole census is 19.4% by script tag, and 17.0% by rendered widget.

On 32 of the 50 stores, the widget's own markup fails the audit

190 of the 1,819 failing instances are inside the overlay itself — content the vendor injected, not the store's code. Almost all are region failures: the injected trigger and screen-reader link sit outside any landmark, so a screen-reader user navigating by landmark passes straight over the accessibility button.

One store's accessiBe trigger is an <li role="button"> wrapping another interactive element — nested-interactive, a control assistive technology cannot reliably operate. That is the button a screen-reader user is meant to press to switch the accessibility features on.

What this does and does not show

Every limitation below cuts against the finding. We would rather publish them than have someone else discover them.

This measures the default page. Some overlay features — screen-reader profiles, keyboard modes, contrast toggles — activate only when a visitor opens the widget and picks a profile. They are not in the delivered DOM, so axe cannot see them, and this study does not claim they are absent. It claims the page as served to every visitor still fails.
Automated testing catches a minority of WCAG. Roughly 30–40% of success criteria are machine-detectable. Every count here is a floor, not a total.
We corrected ourselves once already. The first edition, at n=10, reported overlay prevalence of 33%. At n=294 the true figure is 19.4% by script tag. Small samples mislead, which is the reason for this edition.
Overlay presence showed no association with fewer failures — nor with more. Across all 294 valid sites, median failing instances were 22 with an overlay and 17 without. The gap is small and the direction is unhelpful to both arguments; we report it rather than pick the reading we prefer.
This is not a random sample of the web. It is the client rosters of fifteen US e-commerce agencies, 76% Shopify. It supports claims about these stores, not a population estimate.
Homepages only, one page per store, desktop viewport, light colour scheme. Checkout, collection, product and account templates were not scanned and typically carry more.
19 of the 313 storefronts were excluded, not counted as clean — seven 403s, two 500s, a 404, a 503, a 409, a bot-challenge interstitial, a stub page, and five scan errors. Findings measured on a CDN error page belong to the CDN, not the merchant.
Nothing here is a legal opinion. We are not lawyers and this says nothing about anyone's exposure.

How to reproduce it

Load the page in Chromium at 1366×900. Wait 15 seconds after domcontentloaded. Run axe-core 4.13 with tags wcag2a, wcag2aa, wcag21a, wcag21aa, wcag22aa.

Before quoting a number, do the two things that make it defensible. Count the vendor's rendered nodes to prove the widget is live. Then check whether each failing node carries the vendor's own attributes — conflating the store's defects with the widget's overstates the case by about 10% in this sample.

And discard any page that is not the page: status ≥ 400, a bot-challenge title or challenge DOM, or a body with no interactive elements. Nineteen of our 313 were error pages or interstitials. Counting those as merchant defects is the easiest way to publish something false.

Why we ran this

Tabpath fixes accessibility defects in live e-commerce storefronts. We built the scanner to find work, and this was the first thing it found that we did not expect: a sixth of the stores we looked at had already paid someone to solve the problem, and the problem was still there fifteen seconds after load, with the vendor's own button sitting on the page.

If you sell or install overlays, we will publish a correction on any figure here you can show us is wrong. We have already corrected one of our own.