Skip to content

Stop duplicate CI runs and surface test runner errors - #808

Merged
egiurleo merged 1 commit into
mainfrom
fix-ci-flake-diagnostics
Sep 18, 2026
Merged

egiurleo merged 1 commit into
mainfrom
fix-ci-flake-diagnostics

Conversation

@egiurleo

Copy link
Copy Markdown
Contributor

What's broken

PR #807 had a failing build check that had nothing to do with its dependency bumps. The workflow triggers on on: [push, pull_request], so every in-repo branch (including every Dependabot branch) gets two identical runs. For commit 43a9411, the two runs started three seconds apart:

Run Event Result
34853206151 push resolved + downloaded VS Code 1.137.0, passed
34853212259 pull_request died 290ms into Resolving version...

Same commit, same workflow, opposite results. The duplicate jobs race each other for the same VS Code download and can get throttled. Re-running the failed job passed, confirming the flake.

The failure was also undiagnosable. src/test/runTest.ts caught the error and logged only:

- Resolving version...
Failed to run tests

because console.error("Failed to run tests") discarded err.

Changes

  • .github/workflows/ci.yml — limit push to main, so branch pushes are covered once by the pull_request event. Adds a concurrency group so superseded PR runs get cancelled. cancel-in-progress is conditional on pull_request, so post-merge runs on main are never cancelled and every merged commit keeps its own signal.
  • src/test/runTest.ts — pass err to console.error so the cause and stack trace reach the log.

Verification

  • yarn run lint passes (eslint + prettier, including the reformatted YAML).
  • yarn run test passes locally.
  • Forced a resolution failure against a throwaway copy of the compiled runner to confirm the catch block now surfaces the cause at exactly the point CI went silent:
- Resolving version...
Failed to run tests Error: Invalid version 0.0.0-does-not-exist
    at download (node_modules/@vscode/test-electron/out/download.js:344:23)
    at async downloadAndUnzipVSCode (node_modules/@vscode/test-electron/out/download.js:444:12)
    at async runTests (node_modules/@vscode/test-electron/out/runTest.js:52:40)

Note this PR does not add retry logic for the VS Code download — removing the self-inflicted duplicate run addresses the observed cause, and a real outage should still fail loudly (and now legibly).

🤖 Generated with Claude Code

CI runs on both `push` and `pull_request`, so every in-repo branch
(including Dependabot's) gets two identical runs. On PR #807 the two runs
for 43a9411 started three seconds apart: the push run downloaded VS Code
1.137.0 and passed, while the pull_request run died 290ms into
"Resolving version..." — the duplicates race each other for the same
VS Code download and can get throttled.

Limit `push` to main so branch pushes are covered once by the
pull_request event, and add a concurrency group so superseded PR runs are
cancelled. Post-merge runs on main are not cancelled, so every merged
commit keeps its own signal.

That failure was also undiagnosable: the catch block in runTest.ts logged
"Failed to run tests" and discarded the error, so the log named no cause.
Pass the error to console.error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Assisted-By: devx/a4b3b2ac-0c11-4784-be17-3f07cf65256b
@egiurleo
egiurleo requested a review from a team as a code owner September 18, 2026 14:07
@egiurleo
egiurleo merged commit df8387c into main Sep 18, 2026
3 checks passed
@egiurleo
egiurleo deleted the fix-ci-flake-diagnostics branch September 18, 2026 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants