Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
3fedb1a
M0: hidden `quarto dev-call axe` skeleton command
cwickham Aug 20, 2026
f36c85d
M1: scan an already-rendered site end-to-end
cwickham Aug 20, 2026
2c13f2c
Scan the whole document: drop the axe context excludes
cwickham Aug 20, 2026
72ab8b0
M2: aggregate, report and the two contracts
cwickham Aug 20, 2026
cb5f740
Version the signature scheme, and pin the normalization with tests
cwickham Aug 20, 2026
945aa71
Let Zod's .strict() do the baseline error reporting
cwickham Aug 20, 2026
a1e1590
M3: fixture site, unit tests and a browser smoke test
cwickham Aug 20, 2026
a308250
M4: survive the bundle, and stop importing a browser module to do it
cwickham Aug 20, 2026
011a7a6
M4a: select the author's themes, not just the user's OS preference
cwickham Aug 20, 2026
d01a1f0
Fix: light-only colour-scheme links are not a dark theme
cwickham Aug 20, 2026
a048a2a
Split the axe fixture site into one site per concern
cwickham Aug 20, 2026
0205810
M4b: per-page mode discovery replaces the runtime theme probe
cwickham Aug 20, 2026
7dc7421
Never discover the scanner's own output or vendored libs as pages
cwickham Aug 25, 2026
325d456
B1: a rejected CDP send fails its cell closed, not the whole scan
cwickham Aug 25, 2026
1d2c5b0
Fail closed when the page redirects away from the served site
cwickham Aug 25, 2026
b958f87
Wait for readiness instead of a fixed 500ms settle per cell
cwickham Aug 25, 2026
5292488
Add --exclude: site-relative globs to skip, after --pages
cwickham Aug 25, 2026
eec75ff
Skip redirect stubs at discovery, and harden the scan per review
cwickham Aug 25, 2026
a7af58f
Ship report.md and a generated README; retire the HTML report
cwickham Aug 26, 2026
d894038
report.md: link table rows to occurrences; state detail once
cwickham Aug 26, 2026
62c4061
report.md: drop the redundant rule column
cwickham Aug 26, 2026
c9fbfd4
Remove the systemic/localized label; drop the occurrence detail column
cwickham Aug 26, 2026
80928b6
findings.json: emit anchor-relative paths
cwickham Aug 26, 2026
6df5f67
Self-ignore the _axe-checks artifact dir
cwickham Aug 26, 2026
3317258
scan: collision-proof cell names, percent-encoded page URLs
cwickham Aug 26, 2026
586dcbb
discover: two-mode marker matches the toggle definition, not its name
cwickham Aug 26, 2026
8952e50
axe action: exit through exitWithCleanup
cwickham Aug 26, 2026
1a1a114
Add --fail-on: exit 1 on new findings by impact
cwickham Aug 27, 2026
324d5e8
Await the site server's teardown before exiting
cwickham Aug 27, 2026
74e223f
Close the v1 review's test gaps: flag parsing and exit codes
cwickham Aug 27, 2026
b7ce51c
Write llm-docs/axe-scan-architecture.md; repoint dead references
cwickham Aug 27, 2026
45f7d96
dev-docs/axe-scan.md: experimental how-to
cwickham Aug 27, 2026
5fe0bbf
changelog: quarto dev-call axe (PR number pending)
cwickham Aug 27, 2026
9a1eed1
report.md: backtick code spans instead of <code> raw HTML
cwickham Aug 27, 2026
d556c71
Move axe from dev-call to quarto call
cwickham Aug 27, 2026
50076e5
Test the browser-launch failure path
cwickham Aug 27, 2026
38fda5f
Don't dangle an empty colon when Chrome discovery finds nothing
cwickham Aug 27, 2026
84208ec
Docs: cross-page grouping is a heuristic, not a law
cwickham Aug 27, 2026
a4d07e1
Docs: post-render usage and a sticky PR comment
cwickham Aug 27, 2026
df377d8
Help text: say EXPERIMENTAL, not Prototype
cwickham Aug 27, 2026
322b0ec
changelog: fill in the axe PR number (#14815)
cwickham Aug 27, 2026
66324fe
Docs: how to read and render report.md
cwickham Aug 27, 2026
2955e74
Default narrow viewport: 390x844 -> 320x568 (WCAG's reflow width)
cwickham Aug 27, 2026
c47d0c8
Merge remote-tracking branch 'origin/main' into feat/axe-scan-command
cwickham Aug 27, 2026
7fac2eb
Merge remote-tracking branch 'origin/main' into feat/axe-scan-command
cwickham Aug 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .claude/rules/dev-tools/axe-scan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
paths:
- "src/command/call/axe/**"
- "tests/unit/axe-*.test.ts"
- "tests/smoke/axe/**"
- "tests/docs/axe-scan/**"
---

# Axe Scan Command (`quarto call axe`)

For the scanner's architecture — pipeline, mode discovery, signatures,
baseline semantics, exit codes, and the rationale behind each — see
`llm-docs/axe-scan-architecture.md`.

For how to use the command (flags, baseline workflow, CI recipe), see
`dev-docs/axe-scan.md`.

The signature normalization scheme is pinned by
`tests/unit/axe-signature.test.ts`: changing `normalizeSelector` or
`signatureOf` in a way that re-keys signatures requires bumping
`kSignatureScheme` in `schemas.ts` on purpose.
22 changes: 22 additions & 0 deletions .claude/rules/formats/html-dark-mode.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
paths:
- "src/format/html/format-html-info*"
- "src/format/html/format-html-bootstrap*"
- "src/command/render/pandoc-html*"
- "src/resources/formats/html/templates/quarto-html-*body.ejs"
- "src/core/brand/**"
- "src/command/dev-call/axe/**"
---

# HTML Dark Mode

`formatDarkMode(format)` (`src/format/html/format-html-info.ts`) is the single
predicate for "does this page have a dark mode" — `undefined` means no.
The `data-mode` link attribute measures compiled CSS darkness; it does not
declare the author's slot. The dark slot's stable identity is the
`quarto-color-alternate` class.

For the full picture — configuration surface (`theme`, `brand`/`_brand.yml`),
rendered DOM markers, programmatic mode switching, and the known traps
(light-only brand, light-colored dark slots, `theme: darkly`) — see
`llm-docs/html-dark-mode-architecture.md`.
192 changes: 192 additions & 0 deletions dev-docs/axe-scan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
# Scanning a site with `quarto call axe`

`quarto call axe` scans a rendered Quarto site for accessibility
violations with axe-core. It drives headless Chrome over every page, at
desktop and mobile widths, in each colour mode the page ships. It groups
violations by root cause, compares them against a committed baseline, and
writes a report you can read, commit review comments from, or gate CI on.

**Status: experimental.** The command is hidden (it does not show in
`quarto call` help) and makes no stability promise — flags, artifact shapes,
and semantics can change between prereleases. This page is contributor-facing
documentation; it graduates to quarto.org when the command is unhidden (the
same path the `axe:` render option's docs took).

Prerequisite: a Chromium the scanner can find. `quarto install
chrome-headless-shell` is the reliable route; an installed system Chrome or
Edge also works.

## When to reach for it

The render-time `axe:` option checks the page you are looking at, in your
browser, while you author. This command checks the *site*: every page, both
viewports, light and dark, at audit time or in CI. Reach for it when you
want a site-wide inventory, a regression gate, or output an agent can work
through.

## First scan

```sh
quarto render
quarto call axe _site
```

The scan prints its matrix up front (`43 pages (40 light+dark, 3 default) ×
2 viewports — 166 cells`), then one line per cell, then a findings summary.
Artifacts land at the project root (the nearest `_quarto.yml` at or above
the site dir):

- `_axe-checks/findings.json` — every finding, machine-readable.
- `_axe-checks/report.md` — the human summary, GitHub-flavored markdown.
- `_axe-checks/README.md` — generated docs for the artifacts, including the
baseline how-to. Read this after a scan; it is written for whoever (or
whatever) has only the artifact directory in front of them.
- `_axe-checks/cells/` — raw axe output per page × viewport × mode cell.

`_axe-checks/` ignores itself (`.gitignore` with `*`): it is a disposable
snapshot, not a thing to commit.

### Reading report.md

The report is GitHub-flavored markdown, made to be read where markdown
already renders: your editor's preview, GitHub, or the sticky PR comment
from the CI recipe below. Rendering it is optional. For a standalone HTML
view:

```sh
quarto render _axe-checks/report.md
```

then open `_axe-checks/report.html` in your browser. The output lands
beside the report, inside the self-ignoring artifact directory.

`quarto preview _axe-checks/report.md` does **not** work from inside a
project: `_`-prefixed directories are not project inputs, so preview stops
with `No output created by quarto render report.md`. Use render-then-open,
or pass `--report` a path inside your site source (see the flags table) to
render and preview the report as part of the site.

A finding is one *root cause*, not one element: an alt-less image in a
shared include shows up as one finding with an instance count, not once per
page. Grouping keys on axe's element selector (normalized), so this holds
when the pages describe the element the same way — reliably true for
Quarto's own chrome and for repeated template output, and occasionally
wrong for an anonymous element whose surrounding DOM differs page to page
(axe then picks different selectors, and one cause splits into two
findings). Findings on many pages usually come from a shared source — a
template, the theme, Quarto's own chrome — and one fix clears them all.

## Flags

| flag | default | |
|---|---|---|
| `--pages <globs>` | all `*.html` | comma-separated site-relative globs |
| `--exclude <globs>` | — | skip globs, applied after `--pages` |
| `--max-pages <n>` | ∞ | deterministic cap (sorted, first n) |
| `--viewports <WxH,...>` | `1440x900,320x568` | |
| `--themes <light,dark>` | `light,dark` | filters two-mode pages; one-mode pages always scan once |
| `--timeout <ms>` | `30000` | per-cell budget |
| `--settle <ms>` | `50` | extra delay after the page reports ready |
| `--fail-on <impact>` | off | exit 1 on new findings at/above `minor`/`moderate`/`serious`/`critical` |
| `--report <path>` | `_axe-checks/report.md` | put the report elsewhere, e.g. inside your site source |

The narrow default viewport is 320 CSS px — the width WCAG's reflow
criterion (SC 1.4.10) names, equivalent to 400% zoom on a 1280 px window —
so every rule runs against the reflowed mobile layout.

A subset scan (`--pages`, `--exclude`, `--max-pages`) says so loudly in
every artifact: its counts describe the subset, not the site.

## Running it as a post-render step

The command works as a project `post-render` script, so a full render scans
itself:

```yaml
project:
type: website
post-render:
- quarto call axe _site/
```

The script runs from the project directory after the outputs are written,
so the relative site dir and the artifact anchor resolve exactly as they do
on the command line. Exit codes propagate: without `--fail-on`, findings
never fail the render; with `--fail-on`, a new finding at the threshold —
or an incomplete scan — fails `quarto render` itself, with the scan's error
line in the render output. This runs on every full render of the project,
which adds the scan's runtime to each render.

## The baseline workflow

The first scan of a real site reports findings you will not fix today:
upstream defects, deferred best-practice items, known false positives. The
baseline is the committed ledger of those decisions — `_axe-baseline.json`
at the project root, beside (not inside) `_axe-checks/`.

1. Scan, then fix what you can from `report.md`.
2. For each finding you are accepting instead: copy it out of
`findings.json` into the baseline's `findings` array, trim it, and write
a `note` saying why. The generated `_axe-checks/README.md` documents the
entry shape and the scoping rules (`pages: []` accepts site-wide;
a listed `pages` re-alerts anywhere else).
3. Commit `_axe-baseline.json`.

From then on, reports separate **new** findings from **baselined** ones,
and only new findings can fail CI. A baselined finding that escalates in
impact, or shows up on a page outside its scope, re-alerts as new. Entries
a full-site scan no longer sees are reported as stale — prune them by hand.

There is deliberately no `--update-baseline`: every entry exists because
someone wrote it and said why.

## Exit codes and the CI recipe

| exit | meaning |
|---|---|
| `0` | scan complete; no new findings at/above `--fail-on` (when given) |
| `1` | complete scan, new findings at/above the `--fail-on` threshold |
| `2` | scan incomplete — a cell timed out or errored, no browser, nothing to scan. Takes precedence over 1: an incomplete scan never reads as a pass |

A minimal GitHub Actions gate:

```yaml
- uses: quarto-dev/quarto-actions/setup@v2
- run: quarto install chrome-headless-shell --no-prompt
- run: quarto render
- run: quarto call axe _site --fail-on serious
```

`report.md` is GitHub-flavored markdown, so posting it as a PR comment is
workflow configuration, not tooling. A sticky comment beats a plain one:
it updates in place on each push instead of stacking a new comment per run:

```yaml
- if: always()
uses: marocchino/sticky-pull-request-comment@v2
with:
header: axe
path: _axe-checks/report.md
```

(`if: always()` keeps the comment current when `--fail-on` fails the job;
`header` keys the comment so other sticky comments are untouched. GitHub
caps comment bodies at 65,536 characters; a whole-site report on a large
site can exceed it — trim or attach as an artifact instead.)

## Reading the output as an agent

Point the agent at `_axe-checks/` and let it read the generated `README.md`
first. Finding ids are stable across runs (`image-alt-6e3b76`), so "fix
`image-alt-6e3b76`" is a well-defined instruction, and each finding's
`occurrences[]` carries real selectors and HTML excerpts. Fixes belong in
Quarto *source* (`.qmd`, `_quarto.yml`, `_brand.yml`, theme `.scss`) — never
in the rendered site directory.

## Where the pieces are documented

- How it works and why: `llm-docs/axe-scan-architecture.md`.
- What the artifacts mean, baseline entry shape: the generated
`_axe-checks/README.md` (regenerated every scan, always matches the build
that wrote it).
- Render-time single-page checking: the `axe:` HTML format option.
Loading
Loading