coverage: QNX on-target coverage through the score_coverage gcov backend - #626
dcalavrezo-qorix wants to merge 10 commits into
Conversation
github-actions
Bot
added
bazel
QCC is GCC-based, so QNX coverage uses gcov counters: the tests run in QEMU through score_qnx_unit_tests (run_under_qnx), which brings the .gcda files back; Bazel's own per-test collector turns them into LCOV; score_coverage's gcov reporter (eclipse-score/coverage_tool PR #10) produces the same report as the Linux run. - tools/coverage/BUILD: gcov_reporter_wrapper with backend = "gcov" and the QCC package's gcov. - coverage.bazelrc: coverage:qnx_cov on top of bl-x86_64-qnx (run_under, C++ tests only, Bazel's lcov_merger, the gcov reporter, --instrument_test_targets). - coverage_report_qnx.yml: the QNX build workflow's setup steps (SDP, credential helper, QEMU, KVM) followed by bazel coverage and generate_coverage_html --platform qnx; the artifact is uploaded as baselibs_cpp_coverage_report_qnx. VALIDATION ONLY: MODULE.bazel carries a git_override to the coverage_tool branch; it is replaced by the 0.3.0 registry release once that is published. Rust sources are listed as not-instrumented in the QNX report and measured by the Linux run. Signed-off-by: Dan Calavrezo <195309321+dcalavrezo-qorix@users.noreply.github.com>
951166a to
967596c
Compare
|
Documentation preview for this pull request is available at: |
A wildcard build on a Linux host without QNX credentials must not fetch the QNX SDP package the reporter depends on; --coverage_report_generator names the target explicitly. Signed-off-by: Dan Calavrezo <195309321+dcalavrezo-qorix@users.noreply.github.com>
|
First real QNX run, job https://github.com/eclipse-score/baselibs/actions/runs/36388596870/job/108819220167 (bl-x86_64-qnx, 278 tests in QEMU): artifact |
The gcov backend for QNX on-target coverage is released. Pin the release commit until the registry entry (bazel_registry PR #519) is merged, then drop the override. Signed-off-by: Dan Calavrezo <195309321+dcalavrezo-qorix@users.noreply.github.com>
Signed-off-by: Dan Calavrezo <195309321+dcalavrezo-qorix@users.noreply.github.com>
The release is in the S-CORE Bazel registry; drop the temporary git_override that pinned the release commit. Signed-off-by: Dan Calavrezo <195309321+dcalavrezo-qorix@users.noreply.github.com>
Bazel derives the default --instrumentation_filter from the packages of the test targets and strips only a trailing /tests to reach the code under test. Our tests live in test/ subpackages, so libraries such as //score/os were outside the guessed filter; the gcov collector then dropped their counters and the QNX report showed them at 0 % (13 % for score/os against 80 % on Linux). Instrument everything under //score. Signed-off-by: Dan Calavrezo <195309321+dcalavrezo-qorix@users.noreply.github.com>
…selibs into dcalavrezo_qnx_coverage
0.3.1 requires the explicit --instrumentation_filter this branch already sets, and warns when a report shows the pattern of a library tested from a test/ subpackage without data. Signed-off-by: Dan Calavrezo <195309321+dcalavrezo-qorix@users.noreply.github.com>
dcalavrezo-qorix
changed the title
|
@4og @fbaeuerle could you review this one, please? It moves the coverage pipeline to the qualified |
| # in QEMU through score_qnx_unit_tests, which brings the .gcda counters | ||
| # back. Same-repository branches only: the QNX secrets are not exposed to | ||
| # forks on pull_request events. | ||
| qnx-coverage-report: |
There was a problem hiding this comment.
The generated HTML report completely misses qnx-only targets from this folder https://github.com/eclipse-score/baselibs/tree/main/score/os/qnx
Instead it includes files from score/os/linux which have zero coverage.
I'd expect that both linux and qnx coverage tools produce data only for the code that can be compiled on those platform.
| # back. Same-repository branches only: the QNX secrets are not exposed to | ||
| # forks on pull_request events. | ||
| qnx-coverage-report: | ||
| name: Coverage bl-x86_64-qnx (gcov, on target) |
There was a problem hiding this comment.
The bl-x86_64-qnx got recently renamed to score-qnx-x86_64
| if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository | ||
| runs-on: ubuntu-24.04 | ||
| steps: | ||
| - uses: eclipse-score/more-disk-space@3fac5780033e636b114cf69e891dc19d640855c2 # v1.1 |
There was a problem hiding this comment.
The part below looks identical to https://github.com/eclipse-score/cicd-workflows/blob/main/.github/workflows/qnx-build.yml Can it be somehow reused?
Additionally, this job will be failing in forks due to pull_request trigger
| # into LCOV; score_coverage's gcov reporter produces the report. Rust sources | ||
| # are listed as not-instrumented here and measured by the Linux run. | ||
| # --------------------------------------------------------------------------- | ||
| coverage:qnx_cov --config=bl-x86_64-qnx |
There was a problem hiding this comment.
| coverage:qnx_cov --config=bl-x86_64-qnx | |
| coverage:qnx_cov --config=score-qnx-x86_64 |
This branch has not been deployed
2 participants
Uses the
score_coverage0.3.1 registry release (release notes), which adds the gcov backend for QNX on-target coverage.What
QNX on-target coverage for the C++ tests, through the gcov backend of
score_coverage:score_qnx_unit_tests(run_under_qnx), which brings the.gcdacounters back;score_coverage's gcov reporter applies the coverage scope, adds 0 % baselines from the.gcnonotes, and produces the same report archive as the Linux run (HTML via gcovr, LCOV,unmapped_files.txt, justifications, gate).Changes:
MODULE.bazel:score_coverage0.2.0 → 0.3.1;tools/coverage/BUILD:gcov_reporter_wrapper(backend = "gcov", QCC'sgcov), taggedmanualsobazel build //...on a Linux host does not fetch the QNX SDP;tools/coverage/coverage.bazelrc:coverage:qnx_covconfig (QNX build config,run_under_qnx, Bazel'slcov_merger, gcov reporter,--instrument_test_targets, C++ only);.github/workflows/coverage_report.yml: jobqnx-coverage-report(QNX SDP + QEMU/KVM setup,bazel coverage --config=qnx_cov,generate_coverage_html --platform qnx), artifactbaselibs_cpp_coverage_report_qnx. Same-repository branches only, the QNX secrets are not available to forks.The Linux job is unchanged.
Validation
The QNX job produced a complete report on this branch: 615 files in the index, every link resolves, 81.6 % raw line coverage.
unmapped_files.txtlists 50 Rust sources asnot-instrumented(rustc emits no gcov counters; they are measured by the Linux run), 128 files asno-data(mostly headers vendored from external repositories, which Bazel's collector drops on this backend), 107declaration-onlyheaders and 78compiled-without-codeunits, as on Linux.Per file, the QNX numbers differ from the Linux report of the same tree by design: gcov counts fewer lines (no unused inline functions, no closing braces), so totals differ while no file is missing. The two reports are compared per file, not merged.
Tracking: eclipse-score/tooling#427 (track 1).
Update: bumped to 0.3.1, which requires the explicit
--instrumentation_filter=^//score[/:]this branch sets. Bazel guesses the filter from the packages of the test targets; libraries tested from atestsubpackage were otherwise compiled without counters or, on QNX, lost their counters (score/os showed 13 % instead of 80 % in the first QNX report). See the 0.3.1 release notes.