Repository navigation
ci(windows): measure MSVC coverage with covdbg - #219
liasoft-lia wants to merge 1 commit into
Conversation
covdbg replaces OpenCppCoverage in the MSVC leg, set up with liasoft/setup-covdbg instead of Chocolatey. It writes repository-relative paths, so the path normalization step is gone, and .covdbg.yaml keeps the same selection (examples/, src/ and tools/ without the broker test copy). Pull requests from forks, which get no OIDC token, run the tests without coverage, and CI-Results uploads only the reports that exist. docs/development.md and a comment in tests/CMakeLists.txt name covdbg.
|
|
|
|
Hi! The CLA check is waiting on us. We're contributing as a company (Liasoft GmbH), and your contributing guide says entities can't sign through CLA-assistant and should sign the entity CLA instead. How would you like to receive it: is a signed PDF by email fine, and if so, to which address? Happy to use whatever process you prefer. |
|
@liasoft-lia Thank you for the PR. I think I will go with #220 instead though. Nothing against covdbg, I just don't want to introduce another service, especially for a one off repo of ours (we normally don't use MSVC). In any event, I really appreciate being notified that OpenCppCoverage was archived. |
Description
Hi! This PR switches the MSVC coverage leg from OpenCppCoverage to covdbg. Up front, so there's no surprise: covdbg is made by us at Liasoft, and this PR comes from our team's account. covdbg is a drop-in replacement for OpenCppCoverage, whose repository was archived in September. It reads the test binary and its PDB, so the build stays free of coverage instrumentation, and it's free for public repositories like this one. Its engine is closed source. We prepared it with AI assistance, and a person on our team reviewed every change before opening it.
What gets simpler for you:
--propagate-exit-code), and the Codecov upload with theWindows-MSVCflag is unchanged..covdbg.yamlkeeps the same selection:examples/,src/andtools/, without the broker test hook's private copy.ci.ymlandci-build.ymladdid-token: writenext to the library build'scontents: read. Despite its name,id-token: writegives no write access to the repository: it only lets the job ask GitHub for an OIDC token, which covdbg uses to recognise a public repository, so there's no secret and no sign-up for you.masterand on PRs from branches in this repository. GitHub doesn't hand pull requests from forks the OIDC token covdbg uses to recognise the repository, so fork PRs (this one too) run the MSVC tests as before without coverage, andci-results.ymluploads only the reports that exist.docs/development.mdand a comment intests/CMakeLists.txtnow name covdbg for MSVC coverage.The Codecov badge shows our fork's branch
covdbg-codecov-proofat0852186: this PR's commit plus one fork-only change that lets a fork upload to Codecov. In that run every job passed, and theWindows-MSVCupload lists the same 29 files as the report, under their repository paths.Beyond CI there's an HTML report, coverage in the editor via the VS Code extension or the CLion plugin, and an MCP server so a coding agent can see what the tests still miss. All optional. covdbg runs on Windows today, for 32-bit, 64-bit and ARM64 programs. Linux and macOS support is on our roadmap and already in beta testing; until then gcovr stays for the other legs.
If it's not a fit, feel free to close it; we won't follow up. We'd really appreciate a short note on why, though, and what we could do better.
Screenshot
Issues Fixed or Closed
Roadmap Issues
Testing
Locally, on 45c9b4d (Debug, Visual Studio 2026, no driver installed), both tools over
test_libvirtualhid.exe:.covdbg.yamlSame executable lines in the same files. The one line apart,
windows_backend.cpp:1493, is the Switch Pro report thread's stop branch: it depends on thread timing, and it came out both ways across our runs. Under both tools 171 tests passed and the same 11WindowsConsumerTestcases failed, because the driver isn't installed on that machine; in CI the driver is installed first. On our fork, with the driver installed, the MSVC job passes under covdbg: 191 tests passed and 3 skipped, among them 10 of the 11WindowsConsumerTestcases; the 11th,XInputDoesNotExposeSubmittedXboxBattery, skips itself because XInput doesn't enumerate the virtual gamepad on GitHub's hosted runner. The report has 4729 of 6366 lines covered (74.3%), with repository-relative paths: the run,reports-Windows-MSVCartifact.Type of Change
Checklist
AI Usage
See our AI usage policy.
Claude prepared the workflow change, ran the coverage comparison and drafted this description; a person on our team reviewed all of it before opening the PR.
AI attribution: Claude (agent: Claude Code; model: Claude Opus 5.5,
claude-opus-5-5; thinking level: default).