Skip to content

ci: publish test results from a dedicated Test Results job - #151

Merged
mcollovati merged 1 commit into
mainfrom
ci/test-results-job
Aug 6, 2026
Merged

ci: publish test results from a dedicated Test Results job#151
mcollovati merged 1 commit into
mainfrom
ci/test-results-job

Conversation

@vaadin-bot

Copy link
Copy Markdown
Collaborator

Problem

The Test Results entry was missing from the run UI on fork-validation PRs — e.g. run 31086263314 (PR #150) had no Test Results, while run 30629535865 (PR #148) did.

It was not a config difference — the workflow file was identical at both commits. Test Results was never a job; it was a check run created through the Checks API by publish-unit-test-result-action. That API only takes a head_sha, so GitHub attaches the check run to the oldest github-actions check suite for that SHA.

Fork validation guarantees a collision:

Run 31078116018 PR from TatuLund/browserless-test:gridpro, unit-tests and validation-status skipped by the fork guard — but it still creates check suite 84298004503 for SHA 28d485d
Run 31086263314 Same commit pushed to fork_validation/gridpro → PR #150, own suite 84320399015, full validation runs

The check run published by the second run landed in suite 84298004503 and reported run_id: 31078116018, so it showed up under the skipped fork run instead:

$ gh api repos/vaadin/browserless-test/actions/runs/31078116018/jobs
Format Check       success  06:41:16
Build Project      success  06:41:38
Unit Tests         skipped  06:42:55
Validation Status  skipped  06:42:55
Test Results       success  08:51:06   <-- produced by run 31086263314

PR #148's commit had only one run, hence only one suite, which is why it worked there.

Fix

Move the download/publish steps out of validation-status into their own job named Test Results, and set check_run: false. A job's check run is created by Actions inside its own run's check suite, so the report can no longer be misattributed. Tables go to that job's summary.

  • Branch protection is unaffected — the visible check is still named exactly Test Results, now backed by a job.
  • The if: guard replaces the fork check: on fork PRs unit-tests is skipped, so test-results skips too instead of failing on a missing artifact.
  • fail_on keeps its default, so the job goes red on test failures as before. Deliberately not added to Check all jobs statusunit-tests already gates that, and report publishing shouldn't block a PR.
  • Dropped the now-unused checks: write from validation-status.

Verification

Pushed to fork_validation/gridpro first — run 31088369186 is green with Test Results as a real 5-step job in the run's own suite 84326319601, log confirming 126 JUnit XMLs read and Created job summary, and no stray check run anywhere.

🤖 Generated with Claude Code

A check run created through the Checks API is attached to the oldest check
suite for its head SHA. For fork-validation PRs the head commit already
has a check suite from the fork's own run (where unit-tests is skipped), so
the published "Test Results" check landed under that earlier run and was
missing from the run that actually produced it.

Move the download/publish steps into their own job named "Test Results" and
set check_run: false. A job's check run is created inside its own run's
check suite, so the report now always shows up in the right run. The visible
check name is unchanged, so branch protection is unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mcollovati
mcollovati enabled auto-merge (squash) August 6, 2026 09:28
@mcollovati
mcollovati merged commit 100db76 into main Aug 6, 2026
6 checks passed
@mcollovati
mcollovati deleted the ci/test-results-job branch August 6, 2026 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants