Skip to content

perf(test): add a per-suite timing option and a wall-clock baseline doc with the Linux column #10200

Open

Description

@kyle-sexton

Parent: #3716 (slice 1 of 3). Runs first; slice 2 waits on it. Slice 3 runs in parallel.
Runs in: lane. Linux only; the Windows column is slice 2.

Problem

#3716 criterion 1 asks for a committed baseline table of per-suite wall clock on Windows and on the Linux CI runner. The Linux numbers partly exist: scripts/suite-seconds.txt holds median seconds per suite, but they are inferred from CI log timestamps (its header comment), because scripts/run-plugin-tests.sh prints only PASS: <suite> and no per-suite time. A Windows host has no way to produce comparable numbers, and the gate scripts the issue names (scripts/check-purged-em-dashes.sh, scripts/check-changed-skills.sh, scripts/check-changelog-parity.sh, plugins/skill-quality/scripts/check-skill.sh, plugins/planning/scripts/check-open-questions.sh; #3717 merge comment) have no recorded time at all.

Fix

  1. Add a timing option to scripts/run-plugin-tests.sh (for example --timings <file>) that writes one TSV row per suite: suite path, wall seconds, exit code. Same option, same format, on Linux and Git Bash. No behavior change without the flag.
  2. Add a way to time the gate scripts above in their typical single-diff invocation into the same TSV format (a flag on an existing script or one small script; name it in the doc).
  3. Commit docs/test-wall-clock-baseline.md: how to produce each column, a Linux CI column filled from a whole-tree CI run with the new option (cite the run), the gate-script rows, and an empty Windows column that slice 2 fills. Link it from the header comment of scripts/affected-tests.sh.

Done when

  • bash scripts/run-plugin-tests.sh --timings <tmpfile> <one suite> exits 0 and writes one row with three tab-separated fields
  • test -f docs/test-wall-clock-baseline.md exits 0, and the file cites the CI run behind its Linux column
  • grep -c 'test-wall-clock-baseline.md' scripts/affected-tests.sh prints 1 or more
  • The doc has a row for each of the five gate scripts named above
  • The existing suites pass unchanged (ci-status is green on the PR head)

Split by the backlog HITL session (2026-10-25, owner rule Q93: split large work before a run picks it up). This was generated by AI.

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 26, 2026
  2. github-actions commented on Oct 26, 2026

    @github-actions
    Contributor

    Proposed work class: work-class: scoped
    Basis: automated intake triage; the lanes App applied this class.

    Size: Medium
    Complexity: Low
    Uncertainty: Known
    Flags: none
    Blocked by: none

    Understood: add a --timings <file> option to scripts/run-plugin-tests.sh that writes one TSV row per suite (path, wall seconds, exit code), and add a way to time the five named gate scripts into the same format. Also commit docs/test-wall-clock-baseline.md with a Linux CI column citing a whole-tree CI run, the gate-script rows, and an empty Windows column, linked from scripts/affected-tests.sh. The done-when items are concrete. The CI-run citation needs a real whole-tree run, which may have to happen after the change merges. No duplicates found among the open issues.

    Automated triage, applied by the lanes App.

  3. added
    priority: mediumReal value, no hard deadline; normal backlog flow.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority: mediumReal value, no hard deadline; normal backlog flow.status: readyTriaged, unblocked, and fully specified; eligible to pick up.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

    Type

    Fields

    No fields configured for Task.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions