Automated and keyboard accessibility testing against WCAG 2.2 AA, using Playwright and axe-core, with a written audit of what it found.
axe-core's WCAG rules found 0 violations on every SauceDemo page. The keyboard and manual audit found 7 WCAG failures, including one that stops a keyboard-only user from checking out. (axe's best-practice rule
page-has-heading-one, which the WCAG gate leaves out, does flag F-5.) Read the audit.
Automated scanners catch roughly 30 to 40% of accessibility issues. This repo covers that part, then tests what a scanner can't: whether you can reach things by keyboard, whether focus is visible, where focus goes and comes back to, focus hidden behind overlays, ARIA state and page structure.
| ID | Issue | WCAG | Severity |
|---|---|---|---|
| F-1 | Cart is not reachable by keyboard, which blocks checkout | 2.1.1 Keyboard (A) | Critical |
| F-2 | No visible focus indicator anywhere | 2.4.7 Focus Visible (AA) | Serious |
| F-3 | Menu toggle doesn't expose open/closed state | 4.1.2 Name, Role, Value (A) | Serious |
| F-7 | Focus lands on links hidden behind the open menu | 2.4.11 Focus Not Obscured (AA) | Serious |
| F-4 | Closing the menu drops focus to <body> |
2.4.3 Focus Order (A) | Moderate |
| F-5 | Page titles are <span>s; pages have no headings |
1.3.1 Info & Relationships (A) | Moderate |
| F-6 | Placeholder is the only visible label | 3.3.2 Labels or Instructions (A) | Moderate |
Each finding has evidence, reproduction steps, a recommended fix, and a tracking test. See docs/FINDINGS.md.
F-7: keyboard focus is on the Backpack image link, completely under the open menu.
| Layer | Where | What it proves |
|---|---|---|
| Scanner self-test | tests/scanner-selftest.spec.ts |
The gate rejects planted defects by rule name, so a green run can't be a vacuous pass |
| Scanner validation | tests/w3c-bad.spec.ts |
Catches W3C's broken demo; its repaired twin fails only target-size (F-8) |
| Automated scan | tests/saucedemo.axe.spec.ts |
WCAG 2.2 AA axe scan of each page and its interactive states (errors, open menu) |
| Keyboard & ARIA | tests/saucedemo.keyboard.spec.ts |
Regression guards for what works: tab order, Enter, role="alert", landmarks, ARIA snapshot |
| Known issues | tests/saucedemo.known-issues.spec.ts |
One test per finding pins today's defect; it goes red when the defect is fixed |
- The WCAG 2.2 AA tag set. axe tags each WCAG version separately, so the scan uses
wcag2a, wcag2aa, wcag21a, wcag21aa, wcag22aa. Thebest-practicetag is left out because those rules aren't conformance requirements. - An audit trail on every scan.
expectNoViolations()attaches the full axe JSON to the test report. On failure, the message lists each rule with its impact, help URL and sample nodes. - No rule is disabled. The W3C demo's
target-sizegap (F-8) is pinned as the only rule its repaired pages fail, instead of being switched off. - Checks that axe can't do.
isFocusEntirelyObscured()sampleselementFromPointat the focused element's corners and centre to test SC 2.4.11, which has no axe rule. - No timing-based flakes. Menu tests wait for the slide transition to finish
(
getAnimations()), so focus and geometry are read on a stable DOM, not mid-transition. - Chromium only. axe results depend on the DOM, not the browser. Keyboard focus behavior differs between browsers (Safari skips links on Tab by default), so a single engine keeps the audit deterministic.
npm install
npx playwright install chromium
npm test # everything
npm run test:axe # @axe: automated scans + scanner validation
npm run test:keyboard # @keyboard: keyboard audit
npm run report # open the HTML report (includes axe JSON per scan)import { test } from '@playwright/test';
import { expectNoViolations } from '../src/a11y.js';
test('checkout is accessible with a validation error shown', async ({ page }, testInfo) => {
await page.goto('https://shop.example.com/checkout');
await page.getByRole('button', { name: 'Continue' }).click(); // scan the error state too
await expectNoViolations(page, testInfo, { exclude: ['#third-party-chat'] });
});accessibility-testing/
├── playwright.config.ts
├── src/
│ ├── a11y.ts # axe wrapper: WCAG 2.2 AA tags, scan(), expectNoViolations()
│ ├── keyboard.ts # reachableByTab(), isFocusEntirelyObscured(), describeFocus()
│ ├── saucedemo.ts # login, keyboard menu helpers
│ └── targets.ts # sites under test
├── tests/ # self-test · W3C validation · axe · keyboard · known issues
└── docs/
├── FINDINGS.md # the audit report
└── evidence/ # screenshots referenced by findings
.github/workflows/ci.yml runs lint · format · typecheck, then the
accessibility suite in Chromium. The HTML report is uploaded as an artifact. It runs on every push and
PR, and weekly as well, because both sites under test are live and can change without this repo
changing.
This audit is not a conformance claim. It did not include a screen reader pass, zoom/reflow testing, or mobile testing. See Not covered.