Skip to content

Add traceable pytest black-box coverage scenarios - #20

Draft
AlexanderLanin wants to merge 1 commit into
eclipse-score:mainfrom
etas-contrib:testing/black-box-tests
Draft

AlexanderLanin wants to merge 1 commit into
eclipse-score:mainfrom
etas-contrib:testing/black-box-tests

Conversation

@AlexanderLanin

Copy link
Copy Markdown
Member

Why

The consumer coverage checks run as one shell script, which makes individual scenarios difficult to inspect and gives documentation no per-case pytest traceability. Named pytest cases provide separate JUnit entries with requirement metadata for the LLVM and gcov black-box workflows.

Changes

  • Add 31 named pytest cases for coverage gates, reports, artifacts, source mapping, gcov filtering, and error handling.
  • Share the isolated consumer workspace and collected backend reports through pytest fixtures; keep run_integration_test.sh as a shortcut to the Bazel pytest target.
  • Run integration pytest separately in CI, then combine its JUnit report with unit-test reports for docs-as-code.
  • Preserve the coverage artifact upload and document the pytest test target and scenarios.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run //:license-check

Status: ⚠️ Needs Review

Click to expand output
[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.6.0) and connecting to it...
INFO: Invocation ID: c71b802c-8bf9-435f-bba7-0904af76e1b3
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (13 packages loaded, 10 targets configured)

Analyzing: target //:license-check (79 packages loaded, 10 targets configured)

Analyzing: target //:license-check (91 packages loaded, 10 targets configured)

Analyzing: target //:license-check (148 packages loaded, 1232 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3291 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3291 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3291 targets configured)

Analyzing: target //:license-check (160 packages loaded, 5815 targets configured)

Analyzing: target //:license-check (160 packages loaded, 6549 targets configured)

Analyzing: target //:license-check (160 packages loaded, 6549 targets configured)

Analyzing: target //:license-check (160 packages loaded, 6549 targets configured)

Analyzing: target //:license-check (174 packages loaded, 8614 targets configured)

Analyzing: target //:license-check (177 packages loaded, 11983 targets configured)

Analyzing: target //:license-check (178 packages loaded, 11991 targets configured)

INFO: Analyzed target //:license-check (179 packages loaded, 12118 targets configured).
[11 / 16] Creating runfiles tree bazel-out/k8-opt-exec-ST-d57f47055a04/bin/external/score_tooling+/dash/tool/formatters/dash_format_converter.runfiles [for tool]; 0s local ... (2 actions running)
[14 / 16] JavaToolchainCompileBootClasspath external/rules_java+/toolchains/platformclasspath.jar; 0s disk-cache, processwrapper-sandbox
[15 / 16] Building tools/license.check.license_check.jar (); 0s disk-cache, multiplex-worker
INFO: Found 1 target...
Target //tools:license.check.license_check up-to-date:
  bazel-bin/tools/license.check.license_check
  bazel-bin/tools/license.check.license_check.jar
INFO: Elapsed time: 36.986s, Critical Path: 2.56s
INFO: 16 processes: 12 internal, 3 processwrapper-sandbox, 1 worker.
INFO: Build completed successfully, 16 total actions
INFO: Running command line: bazel-bin/tools/license.check.license_check tools/formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Documentation preview for this pull request is available at:
pr-20: https://eclipse-score.github.io/coverage_tool/pr-20/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant