Skip to content

Skip the document-level coverage thresholds on a SearchRoot-scoped coverage runner run - #938

Merged
drmoisan merged 12 commits into
mainfrom
bug/coverage-runner-scoped-threshold-and-format-928
Sep 29, 2026
Merged

drmoisan merged 12 commits into
mainfrom
bug/coverage-runner-scoped-threshold-and-format-928

Conversation

@drmoisan

Copy link
Copy Markdown
Owner

Suggested title

Skip the document-level coverage thresholds on a SearchRoot-scoped coverage runner run

Summary

  • A coverage runner run that is scoped with -SearchRoot to a directory other than the repository root now skips the two document-level threshold assertions (80 percent line, 75 percent branch) and writes exactly one warning naming the skip and the scoped search root.
  • An unscoped run (omitted -SearchRoot, -SearchRoot ., or -SearchRoot .\) is unchanged: both assertions still run, in the same order, with the same 80 and 75 literals and messages in Invoke-MSTestWithCoverage.Threshold.ps1.
  • A scoped run whose collection fails still terminates with an error.
  • No new command-line switch is added, so no caller can opt out of the unscoped gate.
  • The scoped-run behavior and the definition of a scoped run are documented in the comment-based help of Invoke-MSTestWithCoverageMain.
  • Both coverage runner scripts are formatter-clean under the PoshQC formatter route; the formatter rewrote neither file.

Why

The threshold gate was designed for whole-solution runs. -SearchRoot narrows the executed tests without narrowing the coverage denominator, so a single-assembly run failed the solution-wide floors on a healthy tree. Tracked as issue 928, which consolidates the earlier scoped-threshold report and the formatting residual left by the Dependabot repair work.

What Changed

Core behavior

  • scripts/vscode/Invoke-MSTestWithCoverage.Scope.ps1 (new, 104 lines):
    • Test-CoverageRunIsScoped: a pure predicate. It compares the full paths of the resolved search root and the repository root, ordinal case-insensitive, with trailing separators ignored. It rejects empty and relative inputs.
    • Assert-CoberturaCoverageThresholdForRun: writes one warning and returns on a scoped run; otherwise it runs the two existing threshold assertions unchanged and in order.
  • scripts/vscode/Invoke-MSTestWithCoverage.ps1: dot-sources the new part file, replaces the two unconditional assertion calls with one unconditional call to Assert-CoberturaCoverageThresholdForRun, and adds .DESCRIPTION and .PARAMETER SearchRoot help (+24/-2).

Tests

  • tests/scripts/vscode/Invoke-MSTestWithCoverage.Scope.Tests.ps1 (new, 22 tests):
    • Predicate cases, including a sibling directory whose name extends the root name, case differences, trailing separators, and relative or empty input.
    • Entry-point cases: a scoped run below both floors completes with exactly one warning; a scoped run with a non-zero collector exit throws; unscoped runs below the line or branch floor throw the existing messages; the help text is present.
    • Direct cases for the new gate function.
  • No test uses temporary files or TestDrive. Every filesystem command is mocked, and the real threshold assertions and the real collection wrapper run.

Docs and evidence

  • Minor-audit feature folder with the plan, a remediation cycle, audit artifacts, and evidence under evidence/. Committed evidence carries derived figures only, with <repo-root> placeholders.

Architecture / How It Fits Together

Invoke-MSTestWithCoverageMain resolves the repository root and the search root, collects and post-processes coverage, and then calls Assert-CoberturaCoverageThresholdForRun. That call is the single decision point.

The gate lives in the path-loaded part file rather than in the entry point for a coverage reason. Pester breakpoint coverage credits an entry-point line only from the first test file that parses its own copy of the entry point, so a scoped-arm line written in the entry point was reported as uncovered even though a test executed it. Lines in a part file loaded by path are credited from any test file. The first review cycle found this, and remediation cycle 1 moved the gate.

Verification

Completed (from the committed evidence in the feature folder):

  • Format: the PoshQC formatter over scripts/vscode and tests/scripts/vscode produced identical raw-byte hashes before and after. A liveness control confirmed that the formatter rewrites a deliberately mis-indented line.
  • Analyze: PSScriptAnalyzer reports 0 findings on the entry point, the new part file, and the new test file. The scripts/vscode folder total is unchanged at 13, and Invoke-MSTest.ps1 is unchanged at 2 pre-existing, deliberately retained Write-Host findings.
  • Test: the full CI Pester population (tests/scripts/dependencies and tests/scripts/vscode) passed 342 of 342, with 0 failures.
  • Coverage, measured by a direct Pester 5.6.1 run that mirrors .github/workflows/_pester.yml (breakpoint mode, coverage over scripts/dependencies and scripts/vscode):
    • Population line coverage is 94.53 percent (1626 of 1720), against a baseline of 94.49 percent measured by the identical command.
    • No changed production line is uncovered. The new part file is at 100 percent.
  • Expect-fail runs recorded before each fix: 11 of 14, and 6 of 22 in the remediation cycle, failed as specified. All tests passed afterwards.
  • Acceptance criteria AC1 to AC7 in the issue file are checked off. The final reaudit reports 0 blocking findings.

Recommended:

  • Confirm the required CI checks are green on the PR head.
  • To observe the behavior locally, run scripts/vscode/Invoke-MSTestWithCoverage.ps1 -SearchRoot <test project folder>. It should complete with one "Coverage threshold assertions skipped" warning. -SearchRoot . should still enforce the floors.

Backward Compatibility / Migration Notes

  • No parameter was added or removed. CI and the VS Code task both invoke the runner with -SearchRoot ., which is unscoped, so their behavior is unchanged.
  • The CI coverage workflows are not modified.

Risks and Mitigations

  • Risk: an unscoped run could be misclassified as scoped and skip the gate. Mitigation: the predicate normalizes both paths and compares them exactly (not by prefix). Tests cover the dot and dot-backslash forms, trailing separators, case differences, and a sibling name that extends the root name.
  • Rollback: revert this PR. No data or configuration migration is involved.

Review Guide

  1. scripts/vscode/Invoke-MSTestWithCoverage.Scope.ps1: the predicate and the gate function.
  2. scripts/vscode/Invoke-MSTestWithCoverage.ps1: the dot-source, the single call, and the help text.
  3. tests/scripts/vscode/Invoke-MSTestWithCoverage.Scope.Tests.ps1.
  4. Feature folder evidence, if needed. It is large but mechanical, and it contains the audit trail and remediation record.

Follow-ups

Recorded in the reaudit artifacts as candidates, not changed here:

  • Pester breakpoint coverage under-credits entry-point lines that are reached only by later-sorting test files.
  • A fixture path in an existing coverage-runner test file embeds a developer account name.
  • The coverage runner script has no script-level comment-based help.
  • The bundled PoshQC coverage artifact instruments only the .claude and .codex trees.

GitHub Auto-close

  • None (GitHub validation of closing issues is not available in the PR context; tracked issue: 928)

🤖 Generated with Claude Code

https://claude.ai/code/session_01KNZiXntshsLY8vqqCHUvHm

…lan for scoped coverage threshold and runner formatting

Adds the promoted record, issue.md with a derived Acceptance Criteria section, and the atomic plan that cleared executor preflight in round 4. Preparation only for run bugs-2026-09-28.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com
…ius extraction does not record them as writes

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com
…plan check-offs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01KNZiXntshsLY8vqqCHUvHm
…chRoot-scoped runner run

Adds Test-CoverageRunIsScoped in a new part file, gates the two threshold assertions in Invoke-MSTestWithCoverageMain behind it with one warning, documents SearchRoot scoping in comment-based help, and adds 14 Pester tests. The unscoped run keeps both assertions unchanged. Records Phase 1 evidence P1-T1 to P1-T7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01KNZiXntshsLY8vqqCHUvHm
…off, AC6 open on changed-line coverage

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01KNZiXntshsLY8vqqCHUvHm
…le 1 (AC6 changed-line coverage)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01KNZiXntshsLY8vqqCHUvHm
…art file so every changed line is credited

Adds Assert-CoberturaCoverageThresholdForRun to Invoke-MSTestWithCoverage.Scope.ps1; the entry point makes one unconditional call. Test-CoverageRunIsScoped now rejects empty and relative paths. Adds 8 Pester tests (22 in the suite). Full-population Pester line coverage 94.53 percent against the 94.49 baseline, with no uncovered changed line. AC6 checked off. Records remediation cycle 1 evidence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01KNZiXntshsLY8vqqCHUvHm
…-and-format-928

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01KNZiXntshsLY8vqqCHUvHm
@drmoisan
drmoisan merged commit c4ff0e2 into main Sep 29, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant