KeyTrail

Development verification: The default repeatability check runs four full rounds and a separate four-worker comparison. Final 100-round acceptance and release performance evidence are deferred until the candidate is frozen; protocol and retained results. Ordinary CI checks the real package and clean consumers without claiming final acceptance.

Check dialog, tabs and form-error keyboard journeys in your existing Playwright tests. Declare the expected focus order; KeyTrail presses keys, checks the result and attaches a report showing the first mismatch and the observations that could not run.

Dialog API · Tabs API · Form errors API · CI guard · Read reports · Limits · Performance · Release checklist

Status: dialogs, tabs and form errors, 0.1.0-alpha.6 source/package candidate. No npm publication or GitHub release yet. This milestone implements custom ARIA dialogs, fixed tab lists and declared form-error recovery, plus a CI reporter that requires complete results for every discovered journey test and attempt.

Tab to trigger → Enter → initial focus → Tab cycle → Shift+Tab cycle → Escape
                                                                         ↓
                                                           focus returns to trigger

If the dialog closes but leaves focus on the page body:

D07 FOCUS_MISMATCH expected=open-profile observed=body

The corrected component passes the same contract. The runnable example includes both versions; its intentionally broken fixture is our teaching example, not an upstream bug.

Try the packaged example

Node.js 24 and pnpm 11.25.0 are used for development. The tested Playwright peer version is 1.64.1, with its Chromium 156.0.8078.4. Compatibility.

git clone https://github.com/PerchAnton/keytrail.git
cd keytrail
pnpm install --frozen-lockfile --ignore-scripts
pnpm --dir examples/react-portal install --frozen-lockfile --ignore-scripts
pnpm exec playwright install chromium
pnpm build
pnpm test:package

On Linux, install Chromium's system dependencies with pnpm exec playwright install --with-deps chromium when needed. The package check installs the actual archive into two fresh offline consumers. It checks ESM/CJS imports and types, then runs the vanilla dialog, tabs and form examples and the React portal example. Corrected journeys also run with the installed CI guard; a separate defective React process must exit 1 while preserving its HTML/JUnit reports and incomplete attachment. Dependency installation and the React lockfile preparation use the registry. The actual consumer installation then uses --offline --frozen-lockfile in a separate empty directory; browser tests need only the local fixture server. It leaves artifacts/keytrail-0.1.0-alpha.6.tgz, a development verification record and SHA256SUMS. This checks package usability; final acceptance is a separate explicit gate.

Install that archive into an existing Playwright project:

npm install --save-dev /absolute/path/to/keytrail-0.1.0-alpha.6.tgz @playwright/test@1.64.1

Try the React portal

The React example uses the same dialog contract in StrictMode, with the dialog mounted outside the app root. Compare correct behavior, wrong initial focus and lost return focus, then inspect the first mismatch:

pnpm exec playwright show-report artifacts/consumers/react-portal/reports/teaching/html

See reading reports for the deliberately failing process and its not_run observations. React remains an example dependency, not a library dependency.

Write a contract

import { test } from '@playwright/test';
import { assertDialogJourney } from 'keytrail';

test('profile dialog', async ({ page }, testInfo) => {
  await page.goto('/profile'); // Your application and prepared synthetic state.
  const trigger = { id: 'open-profile', locator: page.getByRole('button', { name: 'Edit profile' }) };
  const name = { id: 'display-name', locator: page.getByLabel('Display name') };
  const close = { id: 'close-profile', locator: page.getByRole('button', { name: 'Close' }) };

  await assertDialogJourney(page, testInfo, {
    id: 'profile-dialog', trigger,
    dialog: { id: 'profile', locator: page.getByRole('dialog') },
    openKey: 'Enter', initialFocus: name, tabOrder: [name, close],
    closeKey: 'Escape', returnFocus: trigger,
  });
});

Use one journey per test, allow time for setup and reporting (the example uses 45 s), and let JourneyAssertionError fail application tests. Do not catch and ignore it. The standalone example provides the actual HTML and configuration; /profile above is an application route, not a service supplied by this package.

Test a tab list

assertTabsJourney checks horizontal/vertical arrow navigation, wraparound, automatic or manual activation, optional Home/End and the final Tab exit. Manual activation checks that focus moves while selection stays unchanged until Enter or Space. It records actual sampling intervals; a gap over 100 ms blocks the check with OBSERVATION_GAP instead of claiming a pass.

See the Tabs API and flow diagram and the runnable manual-tabs example.

Test form error recovery

assertFormErrorsJourney checks the invalid submit, error text and field relationships, the declared focus policy, summary links and keyboard correction. It confirms that all declared errors clear, without including typed values or message text in its own JSON. Choose inputMode: 'type' to exercise character key handlers for printable ASCII. The default insertText mode supports Unicode input; the report identifies the mode.

Invalid submit → messages + IDREF → focus → optional summary links
                                         → keyboard correction → errors cleared

See the Form errors API and flow diagram and the working/broken email example.

Require complete results in CI

Add the reporter last and run the named project as a required CI job:

reporter: [['list'], ['keytrail/ci-guard', { projectName: 'journey' }]],
projects: [{ name: 'journey', testMatch: '**/*.journey.spec.ts' }],

Skipped tests, missing or incomplete reports, and failed attempts that later pass on retry fail the guard. A successful test without a successful journey is insufficient. See setup, diagnostics, boundaries and the flow diagram.

What the result means

Status	Meaning
pass	The declared expectation was observed.
fail	An expectation did not match, such as the return focus.
blocked	A limit or unsupported/ambiguous context prevented a reliable check.
not_run	A planned observation was not attempted after an earlier interruption.

journey-result.json is attached to Playwright's report. Every planned observation remains in the result. A failure takes precedence over blocking; coverage.complete still remains false if any checks are blocked or not run. Counts refer to individual observations, not broad accessibility rules. JSON schema.

When to use it

Evaluate KeyTrail when several dialogs, tab lists or forms need the same declared keyboard checks and consistent diagnostics. Keep ordinary Playwright assertions when a small native test is clearer or your component needs behavior outside the supported contract.

This is a behavioral test helper. Passing does not establish WCAG conformance, screen-reader behavior, pointer blocking or complete accessibility. It does not replace axe-core, manual testing or your component tests. It has no dependency on our separate interaction-check, edit-session or RefMint projects.

Development and evidence

pnpm check
pnpm verify:reliability # Four full rounds, then the four-worker comparison
pnpm exec playwright show-report

Tests cover the specified correct/defective dialogs, detached and replaced nodes, unsupported contexts, bounded entry, scenario deadlines and safe own reports. The verification record separates completed checks from the full 0.1 acceptance criteria. The performance protocol compares equivalent fixed workloads. The full-suite verifier records repeatability and attachment isolation. Alpha.6 adds release correctness regressions. The earlier revision 0300552 completed 100 rounds × 163 tests = 16,300 executions, followed by 163 passing tests on four workers, with no retries or semantic differences. Every raw report and its source identity was independently checked. Earlier local and hosted attempts are retained separately and are not added to this historical total. Six subsequent full rounds at 46515dd are retained as development evidence. At 6a9cc6a, after narrowing manual-window reads, 4 × 167 = 668 sequential checks and 167 four-worker checks passed without retries or semantic differences; retained development record. The private observation store and cached reader change the runtime; the earlier counts do not certify them. At 4abf74c, Linux and macOS ARM passed the complete 169-case suite and package consumers. Intel retained one genuine 370 ms gap, so that exploratory matrix failed. The selected Linux + macOS ARM CI now passes at 63f9edf, including all 169 tests and package consumers on each platform; validated CI record. Intel remains a separate capacity investigation, with its failure retained. The cached reader completed 4 × 169 = 676 local sequential tests and 169 four-worker checks, without retries or semantic differences; independently validated development record. The CI investigation records the earlier gaps and the subsequent passing browser suites, including a separate package-harness failure. Final acceptance and fresh release benchmarks are deferred until the candidate is frozen. Real-world adoption has not been established.

Historical measured overhead (0300552)

All three synthetic workloads meet the 20% target on the recorded runner. Each has 30 measured pairs after five warmups, with the same actions and semantic checks. Manual tabs include required observation windows. See method, limitations and raw samples; this is not a general speed or accessibility claim.

Original code, rules and examples are licensed under Apache-2.0.