rewrite(phase7): batch 2 — add real MC/DC coverage ratchet gate (LLVM) - #168
Merged
Conversation
Part of the v3.0.0 ground-up rewrite (cpp-RCP #129, ROADMAP.md Phase 17), branch rewrite/v3-from-c-rcp. Phase 7 (formal verification + MC/DC + release-pipeline hardening), batch 2: real MC/DC (Modified Condition/ Decision Coverage) evidence in CI. cpp-RCP had zero MC/DC infrastructure before this batch (verified: no mcdc/MCDC/coverage-mcdc references anywhere in .github/workflows/ci.yml). ## What this adds Ported c-RCP's existing `mcdc:` CI job (LLVM 18, real llvm-cov/llvm-profdata condition/decision instrumentation, ratchet-gated -- not a 100% gate), adapted for cpp-RCP's header-only C++ architecture: - CMAKE_C_FLAGS -> CMAKE_CXX_FLAGS, CXX: clang++-18 (not CC:). - -sources scoped to include/rcp/*.hpp (cpp-RCP has no src/ dir -- verified; all real implementation logic lives in headers, not the test files), matching c-RCP's own intent of measuring implementation code, not test code. - `cpfusa coverage --mcdc-file` confirmed to have the same class of bug as c-RCP's `cfusa` equivalent: it parses `covered_true_count`/ `covered_false_count` object keys that real `llvm-cov export -format=text` output never produces (verified directly against both cpp-FuSa's source and real export output -- the schema is a positional array, no such keys anywhere). Followed c-RCP's precedent: the job reads llvm-cov export's own totals.mcdc block directly. ## Floor: 60%, set from a real freshly-measured baseline Not copied from c-RCP's own 93% (that number reflects c-RCP's C codebase's own current state, meaningless for cpp-RCP's). Measured locally end-to-end (instrumented build -> 58/58 tests passing under instrumentation -> profile merge -> llvm-cov export) on Homebrew LLVM 18.1.8 (matching the CI job's clang-18 major version): 313/466 = 67.17%. Cross-checked on LLVM 22.1.8: 426/629 = 67.73% -- close agreement despite a 4-major-version gap, evidence the number reflects the test suite, not toolchain noise. Floor set with ~7-point margin below the clang-18-matched measurement, wider than c-RCP's own 2.3-point margin because this measurement also crosses macOS/arm64 (local) vs Ubuntu 22.04/x86_64 (CI) and Homebrew-built vs apt.llvm.org-built clang-18 -- either could plausibly shift condition-pair counts on a header-only codebase (many decisions live in platform-conditional code, e.g. rcp/l2.hpp's raw-socket paths) more than a same-OS compiler bump would. Provenance written into the gate step's own comment and AUDIT_PACK.md §3/§5. ## Verification Independently re-verified (not just the porting agent's own measurement): reproduced the entire instrumented pipeline myself from scratch on the same Homebrew LLVM 18.1.8 toolchain -- 58/58 tests pass under instrumentation, real llvm-cov export produces a non-empty totals.mcdc block: 315/468 = 67.31%, matching the report's 313/466 = 67.17% almost exactly (trivial build-to-build variance, well within the ~7-point floor margin either way). Directly confirmed the cpfusa --mcdc-file schema mismatch by inspecting both cpp-FuSa's parser source and a real llvm-cov export's actual JSON structure side by side. Validated the new/modified ci.yml job's YAML parses cleanly. Full non-instrumented tree rebuilt from scratch separately: 0 errors, 0 warnings, 58/58 ctest suites pass (100%) -- confirming this batch doesn't affect the normal build path at all. .github/workflows/ci.yml: +192 lines (809 -> 1001). AUDIT_PACK.md: +18 lines (140 -> 158, real-MC/DC provenance paragraph in §3 + new row in the §5 CI Gate Summary table). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Matt <47545907+SoundMatt@users.noreply.github.com>
10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
rewrite(phase7): batch 2 — add real MC/DC coverage ratchet gate (LLVM)
Part of the v3.0.0 ground-up rewrite (cpp-RCP #129, ROADMAP.md Phase 17),
branch rewrite/v3-from-c-rcp. Phase 7 (formal verification + MC/DC +
release-pipeline hardening), batch 2: real MC/DC (Modified Condition/
Decision Coverage) evidence in CI. cpp-RCP had zero MC/DC infrastructure
before this batch (verified: no mcdc/MCDC/coverage-mcdc references
anywhere in .github/workflows/ci.yml).
What this adds
Ported c-RCP's existing
mcdc:CI job (LLVM 18, realllvm-cov/llvm-profdata condition/decision instrumentation,
ratchet-gated -- not a 100% gate), adapted for cpp-RCP's header-only
C++ architecture:
verified; all real implementation logic lives in headers, not the
test files), matching c-RCP's own intent of measuring implementation
code, not test code.
cpfusa coverage --mcdc-fileconfirmed to have the same class of bugas c-RCP's
cfusaequivalent: it parsescovered_true_count/covered_false_countobject keys that realllvm-cov export -format=textoutput never produces (verified directly against bothcpp-FuSa's source and real export output -- the schema is a
positional array, no such keys anywhere). Followed c-RCP's precedent:
the job reads llvm-cov export's own totals.mcdc block directly.
Floor: 60%, set from a real freshly-measured baseline
Not copied from c-RCP's own 93% (that number reflects c-RCP's C
codebase's own current state, meaningless for cpp-RCP's). Measured
locally end-to-end (instrumented build -> 58/58 tests passing under
instrumentation -> profile merge -> llvm-cov export) on Homebrew LLVM
18.1.8 (matching the CI job's clang-18 major version): 313/466 = 67.17%.
Cross-checked on LLVM 22.1.8: 426/629 = 67.73% -- close agreement
despite a 4-major-version gap, evidence the number reflects the test
suite, not toolchain noise. Floor set with ~7-point margin below the
clang-18-matched measurement, wider than c-RCP's own 2.3-point margin
because this measurement also crosses macOS/arm64 (local) vs Ubuntu
22.04/x86_64 (CI) and Homebrew-built vs apt.llvm.org-built clang-18 --
either could plausibly shift condition-pair counts on a header-only
codebase (many decisions live in platform-conditional code, e.g.
rcp/l2.hpp's raw-socket paths) more than a same-OS compiler bump would.
Provenance written into the gate step's own comment and AUDIT_PACK.md
§3/§5.
Verification
Independently re-verified (not just the porting agent's own
measurement): reproduced the entire instrumented pipeline myself from
scratch on the same Homebrew LLVM 18.1.8 toolchain -- 58/58 tests pass
under instrumentation, real llvm-cov export produces a non-empty
totals.mcdc block: 315/468 = 67.31%, matching the report's 313/466 =
67.17% almost exactly (trivial build-to-build variance, well within the
~7-point floor margin either way). Directly confirmed the cpfusa
--mcdc-file schema mismatch by inspecting both cpp-FuSa's parser source
and a real llvm-cov export's actual JSON structure side by side.
Validated the new/modified ci.yml job's YAML parses cleanly. Full
non-instrumented tree rebuilt from scratch separately: 0 errors, 0
warnings, 58/58 ctest suites pass (100%) -- confirming this batch
doesn't affect the normal build path at all.
.github/workflows/ci.yml: +192 lines (809 -> 1001). AUDIT_PACK.md: +18
lines (140 -> 158, real-MC/DC provenance paragraph in §3 + new row in
the §5 CI Gate Summary table).
Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com