Skip to content

Add downstream coverage workflow tests - #22

Draft
AlexanderLanin wants to merge 7 commits into
eclipse-score:mainfrom
etas-contrib:testing/downstream-tests
Draft

AlexanderLanin wants to merge 7 commits into
eclipse-score:mainfrom
etas-contrib:testing/downstream-tests

Conversation

@AlexanderLanin

Copy link
Copy Markdown
Member

Why

Validate score_coverage against the real coverage workflows used by direct downstream consumers, so integration regressions are caught where the tool is used. Make the resulting coverage data visible and downloadable from CI.

Changes

  • Add separate, readable pytest targets for the baselibs and lifecycle coverage workflows, with a local override to this checkout.
  • Publish coverage summaries in each GitHub Actions job summary and retain the HTML, LCOV, and justification reports as downloadable test artifacts. Fail if a report contains no measured source files or covered lines; keep lifecycle's existing 66% gate and baselibs' report-only threshold.
  • Run the jobs on pull requests, merge queue, and pushes to main. Main runs seed per-consumer Bazel caches for later PRs.
  • Exclude baselibs' currently failing vajson_serialize_test from this coverage run; its compact-JSON expectations conflict with the consumer's current pretty-print default.
  • Keep these checks limited to the modern direct consumers baselibs and lifecycle; legacy score_tooling consumers are unchanged.

@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run //:license-check

Status: ⚠️ Needs Review

Click to expand output
[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.6.0) and connecting to it...
INFO: Invocation ID: 862e70b4-65a1-4da6-9228-1dec0f064f61
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (49 packages loaded, 10 targets configured)

Analyzing: target //:license-check (88 packages loaded, 10 targets configured)

Analyzing: target //:license-check (103 packages loaded, 37 targets configured)

Analyzing: target //:license-check (148 packages loaded, 1581 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3291 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3291 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3291 targets configured)

Analyzing: target //:license-check (161 packages loaded, 5738 targets configured)

Analyzing: target //:license-check (166 packages loaded, 9814 targets configured)

Analyzing: target //:license-check (166 packages loaded, 9814 targets configured)

Analyzing: target //:license-check (173 packages loaded, 9866 targets configured)

Analyzing: target //:license-check (173 packages loaded, 9866 targets configured)

Analyzing: target //:license-check (177 packages loaded, 11874 targets configured)

INFO: Analyzed target //:license-check (179 packages loaded, 12118 targets configured).

[14 / 16] [Prepa] JavaToolchainCompileBootClasspath external/rules_java+/toolchains/platformclasspath.jar
[15 / 16] Building tools/license.check.license_check.jar (); 0s disk-cache, multiplex-worker
INFO: Found 1 target...
Target //tools:license.check.license_check up-to-date:
  bazel-bin/tools/license.check.license_check
  bazel-bin/tools/license.check.license_check.jar
INFO: Elapsed time: 37.651s, Critical Path: 2.61s
INFO: 16 processes: 12 internal, 3 processwrapper-sandbox, 1 worker.
INFO: Build completed successfully, 16 total actions
INFO: Running command line: bazel-bin/tools/license.check.license_check tools/formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Documentation preview for this pull request is available at:
pr-22: https://eclipse-score.github.io/coverage_tool/pr-22/

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Coverage artifacts are lost on gate and empty-report failures, limiting diagnosis of the regressions these tests detect.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)
What changed in this PR

Adds CI integration tests that validate score_coverage against the baselibs and lifecycle downstream workflows.

Changes:

  • Adds downstream coverage tests and shared workspace/report helpers.
  • Adds dedicated Bazel targets and required development dependency.
  • Adds a cached matrix workflow with summaries and downloadable artifacts.
File Description
tools/​downstream_tests/​test_lifecycle.py Tests lifecycle coverage with its 66% gate.
tools/​downstream_tests/​test_baselibs.py Tests baselibs coverage with required exclusions.
tools/​downstream_tests/​BUILD Defines separate downstream pytest targets.
tools/​downstream_tests/​_consumer_workspace.py Clones consumers, runs Bazel, and retains reports.
MODULE.bazel Adds the score_tools development dependency.
MODULE.bazel.lock Locks the new dependency metadata.
.github/​workflows/​tests.yml Excludes integration tests from the unit-test job.
.github/​workflows/​downstream_tests.yml Runs downstream tests and uploads their results.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tools/downstream_tests/_consumer_workspace.py Outdated
Comment thread tools/downstream_tests/test_lifecycle.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants