Repository navigation
feat(testing): resolve a browser_driver setting and stop by name when its tool is missing #8539
Copy link
Copy link
Closed
Closed
Feature
Copy link
Labels
agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.
Description
Refs: #7580
Agent Brief
Type: Feature
Summary: Add a browser_driver setting, default playwright-cli, that resolves through the testing plugin's config cascade, and make /testing:run-e2e stop with a named message when the resolved tool is not installed.
Current behavior:
- The testing plugin resolves
/testing:run-e2e's keys withplugins/testing/scripts/resolve-config.sh e2e:e2e_driver(auto | harness | run | playwright | chrome),reuse_running_instanceandfeature_map_dir(resolve-config.sh:50-69, key loop atresolve-config.sh:284-288). Layers, highest first:.claude/testing.local.yaml,docs/conventions/testing.yaml,~/.claude/testing.yaml, then the--uservalue fromuserConfig. e2e_driversays who drives the flow. Nothing names which browser tool aplaywrightdrive uses, and no skill checks that the tool resolves before a run./playwright:checkcheckscommand -v playwright-clionly when someone runs it (plugins/playwright/skills/check/SKILL.md:26).- The testing scan's config loader skips run-e2e's top-level keys by name (
plugins/testing/skills/audit/scripts/adapter-load.awk:311); a new top-level key not in that list is parsed as a scan key. - The config-cascade Implementers row for
testing(run-e2e) lists its keys (docs/conventions/config-cascade/README.md:649). - The gating probe is done (debugging: probe disallowed-tools MCP pattern and large-heap converter #7593, Probe result 2026-10-16, Claude Code 2.1.301): a skill's
disallowed-toolsentrymcp__context7__*ormcp__context7removed that server's tools while a tool from another server stayed allowed; and analyze-profile's converter handled a 510.9 MiB heap in 59 s at 3.73 GiB peak memory.
Desired behavior:
browser_driveris a top-level key of the testing plugin's YAML layers and auserConfigoption, defaultplaywright-cli, scope repo. Its value is the command name of one browser tool (letters, digits,.,_,-); any other value is named on stderr and the key takes its default, the same rulee2e_driverfollows.resolve-config.sh e2eprints abrowser_driver <tab> <value> <tab> <source>line, with the same layer order ase2e_driver./testing:run-e2e, when the driver chosen for the run isplaywright(pinned, or picked byauto), checks that the resolvedbrowser_drivercommand resolves on PATH before the first drive. When it does not, the run stops before driving anything with a message that names the setting, the value, where the value came from, and the two ways forward: install that tool, or setbrowser_driverto a browser tool the user has. The message says other browser tools are not used as a silent substitute. The run never switches tools partway through.- The check is a script (or a
resolve-config.shmode) the skill calls, so a test can drive it; the skill text gains the step and does not remove any existing text. testing.schema.json, the plugin'suserConfig,adapter-load.awk:311's skip list,plugins/testing/skills/run-e2e/context/e2e-config.mdand the Implementers row atdocs/conventions/config-cascade/README.md:649all name the new key.
Known call (2026-10-20):
- The PATH check runs whenever the driver chosen for the run is
playwright, whether pinned or picked byauto(autoranks harness, then run, then playwright:plugins/testing/skills/run-e2e/context/e2e.md:85-95). run-e2e/SKILL.mdedits are added lines only. Leave line 139 ("Five keys govern this run") and theresolve-config.sh e2e --user ...command line below it intact, and add thebrowser_driverresolution as new lines (for example a separate--user 'browser_driver=${user_config.browser_driver}'call). Changing a word in an existing line counts as a deleted line.- The Implementers-row edit touches the location cell only, so the generated glance table does not change and is not regenerated (only the Who wins and Merge form cells render, per
scripts/sync-config-cascade-semantics.py's docstring). - "Scope repo" means a repository-scope surface (
docs/conventions/config-cascade/README.md:354). - Test suites and CI-only gates are verified by the PR's CI, not run in the lane.
- Uncertainty is Known: the three open questions (whether
autocounts, whether the glance table regenerates, what scope repo means) have cited answers above, per poteto's slice research of 2026-10-20.
Key interfaces:
plugins/testing/scripts/resolve-config.sh(e2emode) andplugins/testing/scripts/resolve-config.test.sh.plugins/testing/.claude-plugin/plugin.json(userConfig.browser_driver),plugins/testing/schemas/testing.schema.json.plugins/testing/skills/run-e2e/SKILL.md(the step that resolves the five keys, aroundSKILL.md:139-165) andcontext/e2e-config.md.plugins/testing/skills/audit/scripts/adapter-load.awk:311.docs/conventions/config-cascade/README.mdImplementers row (location cell only; the glance table is unchanged).
Acceptance criteria:
-
resolve-config.test.shhas cases for: no layer sets the key, so it printsbrowser_driver,playwright-cli,default;docs/conventions/testing.yamlsets it, so that file is the source; an invalid value is named on stderr and the default stands. -
resolve-config.test.shhas a case wherebrowser_drivernames a command not on PATH and the check exits non-zero with a message containingbrowser_driver, the command name and the source; and a control case where the command is a stub on PATH and the check exits 0. - The PR's CI runs
resolve-config.test.sh(test-bash) and it passes. - Each of
plugins/testing/skills/run-e2e/SKILL.md,plugins/testing/schemas/testing.schema.json,plugins/testing/.claude-plugin/plugin.json,plugins/testing/skills/audit/scripts/adapter-load.awkanddocs/conventions/config-cascade/README.mdmentionsbrowser_driver. - The PR deletes no line from
plugins/testing/skills/run-e2e/SKILL.md. -
run-e2e'sevals/evals.jsongains one case wherebrowser_driverresolves to a missing tool and the expected output is the named stop with no drive attempted. - The PR's CI runs
python3 scripts/sync-config-cascade-semantics.py --checkandscripts/check-convention-yaml.sh --check, and both pass. -
scripts/check-changelog-fragments.sh --checkpasses, with oneminorfragment fortesting. The PR's CI runsscripts/check-changelog-fragments.sh --check-required origin/mainand it passes. - CI's
test-bash(resolve-config suite),lint-repo(cascade glance table, convention YAML) andcheck-skills(skill gate, eval sets) pass.
Out of scope:
- The
app-controlplugin, the capture move andrun-e2e's shrink (app-control: new plugin plus testing and debugging changes #7581). - A profiling tool setting and analyze-profile's
compare(app-control: new plugin plus testing and debugging changes #7581's debugging slice; the probe's memory figure goes there). - Adding
disallowed-toolsto any skill; the probe shows it works, and using it is a later choice. - Installing any tool.
- Regenerating the config-cascade glance table.
Origin:
- One named tool per job, no MCP handoff; the skill names the command, checks it resolves and stops if missing: hers (Lauren's Playwright plus CDP) and the owner's ("follow Lauren or better"), Q561, D55, D116.
- A setting with a named default read through the cascade; use the tool the user has, one per run, never switch mid-run: owner's, Q569. This brief meets "use the tool the user has" by naming the setting in the stop message instead of switching tools on its own.
- Setting scope repo: ours, from the thread's Key interfaces.
- Known call (check on the chosen driver, added lines only, no glance-table regeneration): poteto's slice research, 2026-10-20, recorded by Issue Triage.
Do not run with: #7581 slices (testing and app-control), and any Phase 2 slice still open that edits plugins/testing/skills/audit/ (PLAN's file-overlap matrix pairs Phases 2 and 13).
Slice of #7580, drafted by the Issue Triage session from the thread's recorded decisions.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.
Type
Fields
No fields configured for Feature.
Activity
melodic-automation-lanes commentedon Oct 20, 2026
The lanes App's pull request #8720 links this issue with
Refs:, notCloses, so the merge lane never merges it. report-outcome addedneeds-humanso a person merges or closes it. When it merges, work-unblock removes thisneeds-humanandagent-readyand addsneeds-triage, so triage decides what work is left. To keep the issue held after the merge, addneeds-owner-decision.kyle-sexton commentedon Oct 20, 2026
Delivered by #8720 (merged), which referenced this issue with Refs: rather than a closing keyword.