Skip to content

Require the instrumentation filter; warn when it was left to Bazel - #12

Merged
dcalavrezo-qorix merged 2 commits into
mainfrom
dcalavrezo_instrumentation_filter
Sep 29, 2026
Merged

dcalavrezo-qorix merged 2 commits into
mainfrom
dcalavrezo_instrumentation_filter

Conversation

@dcalavrezo-qorix

Copy link
Copy Markdown
Contributor

Why

baselibs' first QNX report (eclipse-score/baselibs#626) showed score/os at 13 % line coverage against 80 % on Linux, although the tests had run on the target. Cause: Bazel computes the default --instrumentation_filter from the packages of the test targets and strips only a trailing /tests (InstrumentationFilterSupport.java). baselibs tests score/os from score/os/test, so the guessed filter contains ^//score/os/test[/:] and not score/os.

Consequences, reproduced in the integration workspace:

  • gcov backend: Bazel's own collector converts counters only for the targets inside the filter. The score/os counters came back from QNX and were dropped on the host. The files still show up at 0 % because rules_cc had compiled them with counters (a direct dep, futurecpp, is instrumented) and the reporter baselines every notes file.
  • Both backends: a library without an instrumented direct dependency is compiled without instrumentation and appears as no-data. The 0.3.0 docs claimed the filter has no effect; that was true only by the accident of dependencies.

What

  • User manual: --instrumentation_filter=^//<root>[/:] added to step 4 (LLVM config) and required in step 4b (gcov config), with the explanation. Architecture design decision and the known-problems row corrected.
  • Reporters: both warn when in-scope files without test data sit in a directory that is tested from a test/ or tests/ subdirectory, and name the flag. New tool_req__coverage_instrumentation_hint, potential error ERR-13 (under-reporting, detected). Two unit tests.
  • Integration workspace: //lib:cross_pkg, a dependency-free library tested from //lib/test (the baselibs layout), in both goldens with hand-derived numbers. Both configs set the filter. A gcov run with Bazel's guessed filter checks that the file falls to 0 % and that the reporter prints the hint.
  • Correction: with the filter, the header vendored from an external repository is measured on gcov as well (the manifest lists the sources of instrumented targets, external headers included). The gcov golden gains that record; manual and release notes corrected.
  • Release notes 0.3.1; 0.3.0 dated 2026-09-28.

Verification

Check Result
bazel test //score_coverage/tests/... //tools:all 22 targets pass (reporter_test 71 cases)
integration_tests/run_integration_test.sh 25 checks pass, both goldens match
bazel build --config=lint //score_coverage/..., bazel run //tools:format.check clean
bazel run //docs:docs_check build succeeded, 0 schema warnings

Consumer side: the one-line fix is already on baselibs#626 and communication#1171 (QNX configs); logging#338 and lifecycle#715 get the LLVM line as a follow-up commit.

baselibs' first QNX report showed score/os at 13 % against 80 % on
Linux although the tests had run. Bazel guesses --instrumentation_filter
from the packages of the test targets and strips only a trailing /tests;
libraries tested from a "test" subpackage are outside the guess. On the
gcov backend Bazel's collector then drops their counters; on both
backends a library without an instrumented direct dependency is compiled
without counters at all.

- User manual: --instrumentation_filter=^//<root>[/:] is part of the
  LLVM config (step 4) and required for the gcov config (step 4b).
  Architecture and known problems corrected: the filter is not
  irrelevant, it must name the module.
- Both reporters warn when in-scope files without test data sit in a
  directory that is tested from a test/ or tests/ subdirectory and name
  the flag (tool_req__coverage_instrumentation_hint, ERR-13).
- Integration workspace: //lib:cross_pkg tested from //lib/test, in both
  goldens; both configs set the filter; a gcov run with the guessed
  filter checks the warning. With the filter the vendored external
  header is measured on gcov too; golden and docs corrected.
- Release notes 0.3.1; 0.3.0 dated.

Signed-off-by: Dan Calavrezo <195309321+dcalavrezo-qorix@users.noreply.github.com>
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run //:license-check

Status: ⚠️ Needs Review

Click to expand output
[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.6.0) and connecting to it...
INFO: Invocation ID: fcb88121-5396-42c6-a1d7-e46e7bf5b239
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (5 packages loaded, 6 targets configured)

Analyzing: target //:license-check (15 packages loaded, 10 targets configured)

Analyzing: target //:license-check (88 packages loaded, 10 targets configured)

Analyzing: target //:license-check (97 packages loaded, 38 targets configured)

Analyzing: target //:license-check (148 packages loaded, 1570 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3138 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3138 targets configured)

Analyzing: target //:license-check (151 packages loaded, 3138 targets configured)

Analyzing: target //:license-check (162 packages loaded, 9446 targets configured)

Analyzing: target //:license-check (166 packages loaded, 9474 targets configured)

Analyzing: target //:license-check (166 packages loaded, 9474 targets configured)

Analyzing: target //:license-check (173 packages loaded, 11602 targets configured)

INFO: Analyzed target //:license-check (178 packages loaded, 11661 targets configured).
[11 / 16] checking cached actions
[12 / 16] [Prepa] Generating Dash formatted dependency file ...
[14 / 16] checking cached actions
[14 / 16] [Prepa] JavaToolchainCompileBootClasspath external/rules_java+/toolchains/platformclasspath.jar
INFO: Found 1 target...
Target //tools:license.check.license_check up-to-date:
  bazel-bin/tools/license.check.license_check
  bazel-bin/tools/license.check.license_check.jar
INFO: Elapsed time: 39.051s, Critical Path: 2.24s
INFO: 16 processes: 12 internal, 3 processwrapper-sandbox, 1 worker.
INFO: Build completed successfully, 16 total actions
INFO: Running command line: bazel-bin/tools/license.check.license_check tools/formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

The label precedes a paragraph, not a heading, so the bare :ref: had no
caption and the docs job (warnings as errors) failed.

Signed-off-by: Dan Calavrezo <195309321+dcalavrezo-qorix@users.noreply.github.com>
@dcalavrezo-qorix
dcalavrezo-qorix merged commit b7b9de3 into main Sep 29, 2026
9 checks passed
@dcalavrezo-qorix
dcalavrezo-qorix deleted the dcalavrezo_instrumentation_filter branch September 29, 2026 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants