Skip to content

user-experience: install-test and confirm the unconfirmed and deferred route rows #6667

Open

Description

@kyle-sexton

Refs: #6306

Problem

The user-experience plugin ships its route rows (plugins/user-experience/reference/routing.json) without install-testing the third-party tools they name. Per the routing-as-data convention (docs/conventions/routing-as-data/README.md), account-free rows that were not tested ship unconfirmed and account-bound rows ship deferred, so shape and the job skills never route to a deferred row and disclose an unconfirmed one. These rows stay that way until someone installs each tool and confirms the route works.

Job Id Kind Account Status As of
research-instruments /design:user-research skill none unconfirmed 2026-10-09
research-instruments usertesting mcp paid deferred 2026-10-09
synthesis /product-management:synthesize-research skill none unconfirmed 2026-10-09
synthesis /design:research-synthesis skill none unconfirmed 2026-10-09
synthesis dovetail mcp login deferred 2026-10-09
synthesis miro@claude-plugins-official plugin login deferred 2026-10-09
flows-ia figma@claude-plugins-official plugin login deferred 2026-10-09
flows-ia miro@claude-plugins-official plugin login deferred 2026-10-09
flows-ia optimal mcp login deferred 2026-10-09
evaluation /design:design-critique skill none unconfirmed 2026-10-09
evaluation maze mcp login deferred 2026-10-09
journeys miro@claude-plugins-official plugin login deferred 2026-10-09
measurement /product-management:metrics-review skill none unconfirmed 2026-10-09
measurement contentsquare mcp login deferred 2026-10-09
analytics amplitude@claude-plugins-official plugin login deferred 2026-10-09
analytics posthog@claude-plugins-official plugin login deferred 2026-10-09
analytics mixpanel mcp login deferred 2026-10-09

Known weak spots: the optimal row's pointer is the vendor homepage and its detect name is unconfirmed; the test fixture tests/fixtures/mcp-list-empty.txt is not recorded claude mcp list output.

What install-and-test must show, per row

  • The tool installs (plugin from its marketplace, MCP server from its documented setup) and scripts/detect.mjs reports its id under installed.
  • The route's job runs through it from the matching skill (/user-experience:shape or the job skill) and returns output the skill can label.
  • The row's detect, pointer, as_of and recheck are correct as of the test date.

Then set the row to confirmed (or remove it), update as_of, and note the test in the PR.

Acceptance criteria

  • Every row in the table is confirmed with a fresh as_of, or removed with a reason.
  • node --test 'plugins/user-experience/tests/*.test.mjs' passes.

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 9, 2026
  2. github-actions commented on Oct 9, 2026

    @github-actions
    Contributor

    Proposed work class: work-class: scoped
    Basis: automated intake triage; the lanes App applied this class.

    Size: Small
    Complexity: Low
    Uncertainty: Knowable
    Flags: none
    Blocked by: none

    Decision recorded. The owner approved applying independently verified recommendations (confirmed, low-risk set) on 2026-10-13.

    • Authorize an agent to install the plugins behind the 5 account-free skill rows (/design:, /product-management:) in a throwaway config directory and confirm them? → Yes, sandboxed config dir, no user-scope change
      Basis: AGENTS.md 'When to stop' (user-scope config needs the user); docs/conventions/routing-as-data/README.md:40-41
    • What happens to the 12 login or paid rows (usertesting, dovetail, miro x3, figma, optimal, maze, contentsquare, amplitude, posthog, mixpanel)? → Keep them deferred and narrow the acceptance criteria to the account-free rows
      Basis: docs/conventions/routing-as-data/README.md:40-41 ('A deferred route is never used'); plugins/user-experience/reference/routing.json (12 deferred rows as of 2026-10-09)

    [quote truncated; the full decision is at the link]

    #6667 (comment)

    The decision settles the open questions. The work is to install-test the 5 account-free skill rows in a sandboxed config directory and confirm them. The 12 account-bound rows stay deferred.

    Automated triage, applied by the lanes App.

  3. added theissue type on Oct 12, 2026
  4. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 12, 2026
  5. kyle-sexton commented on Oct 13, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded. The owner approved applying independently verified recommendations (confirmed, low-risk set) on 2026-10-13.

    • Authorize an agent to install the plugins behind the 5 account-free skill rows (/design:, /product-management:) in a throwaway config directory and confirm them? → Yes, sandboxed config dir, no user-scope change
      Basis: AGENTS.md 'When to stop' (user-scope config needs the user); docs/conventions/routing-as-data/README.md:40-41
    • What happens to the 12 login or paid rows (usertesting, dovetail, miro x3, figma, optimal, maze, contentsquare, amplitude, posthog, mixpanel)? → Keep them deferred and narrow the acceptance criteria to the account-free rows
      Basis: docs/conventions/routing-as-data/README.md:40-41 ('A deferred route is never used'); plugins/user-experience/reference/routing.json (12 deferred rows as of 2026-10-09)

    routing.json ships 5 unconfirmed account-free skill rows and 12 deferred account-bound rows that need install-testing before they can be confirmed. Account-bound rows need the owner's logins (one is paid), and the acceptance criteria currently demand every row be confirmed or removed.

    Verified by an independent fresh-context check (confirmed, risk low): gh issue view 6667 (OPEN); origin/main docs/conventions/routing-as-data/README.md:40-41 status rules; origin/main plugins/user-experience/reference/routing.json: 5 unconfirmed account-free, 12 deferred (miro x3), 2 confirmed; anthropics/knowledge-work-plugins: PUBLIC, marketplace lists design and product-management; all 5 SKILL.md paths exist (assumption true)

    Admission to an agent lane stays with the work-class rules and owner ratification.

  6. github-actions commented on Oct 25, 2026

    @github-actions
    Contributor

    The triage sweep dispatched intake triage for this issue (attempt 2 of 2).

  7. added
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.
    status: needs-decisionAwaiting a human or maintainer judgment call.
    and removed
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    on Oct 25, 2026
  8. melodic-automation-lanes commented on Oct 25, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane: no trusted-human acceptance criteria: a listed human must put the confirmed - [ ] criteria in the body or post them as their own comment, or, on an issue a listed human filed, intake's newest refinement comment must hold them unedited. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

  9. melodic-automation-lanes commented on Oct 25, 2026

    @melodic-automation-lanes
    Contributor

    The latest triage verdict holds nothing: the issue is at status: ready.

  10. melodic-automation-lanes commented on Oct 25, 2026

    @melodic-automation-lanes
    Contributor

    Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

    Intended outcome: Install-test the design and product-management plugins in a throwaway config directory and set the 5 account-free unconfirmed skill rows in routing.json to confirmed with a fresh as_of, leaving the 12 login or paid rows deferred.

    Acceptance criteria

    • In plugins/user-experience/reference/routing.json the rows /design:user-research, /product-management:synthesize-research, /design:research-synthesis, /design:design-critique and /product-management:metrics-review each have status confirmed, or are removed with a reason stated in the PR.
    • Each confirmed row has as_of set to the test date, not 2026-10-09.
    • The install ran with a throwaway CLAUDE_CONFIG_DIR, and no user-scope config was changed.
    • After install, scripts/detect.mjs reports design@​knowledge-work-plugins and product-management@​knowledge-work-plugins under installed in that config directory.
    • Each of the 5 skills ran from /user-experience:shape or its job skill and returned output the skill can label; the PR notes this per row.
    • Each confirmed row's detect, pointer and recheck were checked against the installed plugin and the marketplace as of the test date, and corrected if wrong.
    • The 12 rows with account login or paid (usertesting, dovetail, miro x3, figma, optimal, maze, contentsquare, amplitude, posthog, mixpanel) are unchanged and still deferred.
    • node --test 'plugins/user-experience/tests/*.test.mjs' passes.
    • The PR description notes the test date and the install steps used, and says the issue stays open for the 12 deferred rows.

    Dependencies

    • Refs #​6306 (parent tracking); no blocking work found.
    • The 12 deferred rows wait on the owner's logins and a paid account, and are out of scope here per the recorded decision.

    Related work

    Sources read

    • /home/runner/work/_temp/refine-data/context.json: kyle-sexton's Decision recorded comment authorizes sandboxed install for the 5 account-free rows and keeps the 12 account-bound rows deferred, narrowing the acceptance criteria.
    • plugins/user-experience/reference/routing.json:5,8,9,17,24: the five account-free skill rows are unconfirmed with as_of 2026-10-09.
    • plugins/user-experience/reference/routing.json:6,10,11,13-15,18,22,25,27-29: the twelve account-bound rows are deferred.
    • docs/conventions/routing-as-data/README.md:40-41: a deferred route is never used; an unconfirmed route is disclosed; an account-bound row is never unconfirmed.

    Automated refinement by the lanes App. Admission counts only criteria a listed human wrote: to confirm these, post them as your own comment whose first line is **Decision recorded.**, which sends the issue back through intake.

  11. kyle-sexton commented on Oct 26, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI.

    Releasing the hold: no question is open. The latest status: needs-decision came from admission ("no trusted-human acceptance criteria"), which ran before the refinement comment holding the criteria was posted. This issue was filed by a listed human, so the unedited refinement counts as its spec (owner Q78). The label comes off and intake triage is dispatched again so admission reads the refinement.

    One correction to the refinement: drop the last criterion's clause saying the issue "stays open for the 12 deferred rows". The 2026-10-13 decision narrowed the acceptance criteria to the 5 account-free rows, so the issue closes when those 5 are confirmed. The 12 paid rows stay deferred (docs/conventions/routing-as-data/README.md:40-41).

  12. added
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    and removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Oct 26, 2026
  13. melodic-automation-lanes commented on Oct 26, 2026

    @melodic-automation-lanes
    Contributor

    The work-resolve-issue lane stopped partway through this issue and needs a person.

    Why, in the run's own words:

    This lane cannot install-test anything, so it changed nothing. The issue as narrowed by the 2026-10-13 decision needs three things: install the design and product-management plugins from knowledge-work-plugins into a throwaway CLAUDE_CONFIG_DIR, run plugins/user-experience/scripts/detect.mjs to show both under installed, and run each of the 5 skills from /user-experience:shape or its job skill. The lane's Bash runs only its named wrapper scripts. It cannot run the claude CLI, node, or a Claude session with those plugins loaded. Setting the 5 rows to confirmed without that test would go against docs/conventions/routing-as-data/README.md:40-41 and the routing.json note, which define unconfirmed as 'has not been install-tested'. The test needs a run with the claude CLI and node available in a sandboxed config directory, either a person or a lane that allows it.

    What the run says it did not do:

    • Install designknowledge-work-plugins and product-managementknowledge-work-plugins into a throwaway CLAUDE_CONFIG_DIR. Not possible: this lane's Bash cannot run the claude CLI.
    • Run plugins/user-experience/scripts/detect.mjs against that config directory and confirm both plugins appear under installed. Not possible: this lane cannot run node.
    • Run each of the 5 skills from /user-experience:shape or its job skill and check the output can be labeled. Not possible: this needs a Claude session with the plugins loaded.
    • Set the 5 rows to confirmed with a fresh as_of and re-check their detect, pointer and recheck. Not done, because the install test above never ran.
    • node --test 'plugins/user-experience/tests/*.test.mjs'. Not run: this lane cannot run node, and nothing changed.
    • PR prep review, verify and checks steps. Skipped: there is no diff to review, build or lint.

    Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40261911416

    To retry, remove needs-human and add agent-ready again.

  14. kyle-sexton commented on Oct 26, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI.

    Routed to the host runner, not the owner. The remaining work is an install test in a throwaway CLAUDE_CONFIG_DIR with the claude CLI and node, which the issue lane cannot run (https://github.com/melodic-software/claude-code-plugins/actions/runs/40261911416).

    Under the owner's 2026-10-26 stop rule, a person is needed only for a change in intent or vision, an implementation finding that changes the intent or primary design, or a high or critical security issue; host, attended and eval runs are agent work. So needs-human comes off. It moves to the host-run label, and the host runner takes it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedMaintainer would welcome help.needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.priority: mediumReal value, no hard deadline; normal backlog flow.status: readyTriaged, unblocked, and fully specified; eligible to pick up.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

    Type

    Fields

    No fields configured for Task.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions