Skip to content

ci: add -Dcoverage_tool=gcov|llvm option, wire native llvm-cov in matrix#740

Open
sublimator wants to merge 18 commits intodevfrom
coverage-llm
Open

ci: add -Dcoverage_tool=gcov|llvm option, wire native llvm-cov in matrix#740
sublimator wants to merge 18 commits intodevfrom
coverage-llm

Conversation

@sublimator
Copy link
Copy Markdown
Collaborator

Summary

Adds a -Dcoverage_tool=gcov|llvm CMake option so we can use native LLVM source-based coverage (-fprofile-instr-generate -fcoverage-mappingllvm-profdatallvm-cov) as an alternative to the existing gcov + gcovr pipeline. Default stays gcov for backwards compat.

Motivation: gcov data is racy under parallel test runs (cf. #737 "suspicious gcov hit counts"). Native llvm-cov is faster, has better branch/region data, and doesn't share that failure mode.

CMake

  • RippledSettings.cmake — new coverage_tool cache var (gcov | llvm) with validation
  • RippledInterface.cmake — split coverage compile/link flags by tool via genex
  • RippledCov.cmake — dispatch to the new helper when coverage_tool=llvm
  • cmake/CodeCoverageLLVM.cmake (new) — setup_target_for_coverage_llvm() driving profraw → profdata → export with format-aware output (lcov / json / txt / html)

CI

  • New clang-20 llvm-cov coverage matrix row (apt.llvm.org bootstrap for clang >= 19 since 24.04 default repos cap at clang-18)
  • Conditional install of llvm-N vs gcovr based on coverage_tool
  • Coverage artifact + Codecov upload generalised to coverage.lcov | coverage.xml; Codecov uploads tagged with flags: ${{ matrix.coverage_tool }} so the streams stay distinguishable
  • Coverage cmake-args switched to *_FLAGS_DEBUG so the build action's -stdlib=libc++ flag isn't clobbered

TEMP — revert before merging to dev

  • Matrix narrowed to just the new llvm-cov row on non-main refs only so we can iterate without burning runners on the rest
  • coverage-llm added to the push trigger so direct pushes to the branch fire CI

Both temporary changes are guarded so they cannot affect dev/candidate/release runs, but they should still be reverted before merge.

Test plan

  • CI run on this branch passes the new clang-20 llvm-cov coverage row end-to-end
  • Codecov accepts the coverage.lcov upload (Codecov v5 with flags: llvm)
  • Verify the produced coverage.lcov actually contains xrpld//libxrpl/ entries (not just empty / test-excluded coverage)
  • Sanity-check report numbers vs. the gcc-13 gcovr report for major modules
  • Revert the two temp items (matrix narrowing + push trigger entry) before flipping out of draft

introduces native llvm source-based coverage as an alternative to the
existing gcov + gcovr pipeline. driven by a new -Dcoverage_tool cache
variable (default 'gcov' for backwards compat; 'llvm' enables
-fprofile-instr-generate / -fcoverage-mapping + llvm-profdata + llvm-cov).

cmake:
- RippledSettings.cmake: add coverage_tool with validation
- RippledInterface.cmake: split coverage compile/link flags by tool
- RippledCov.cmake: dispatch to the new helper when tool=llvm
- CodeCoverageLLVM.cmake (new): setup_target_for_coverage_llvm() driving
  profraw -> profdata -> export with format-aware output (lcov/json/txt/html)

ci:
- new clang-20 llvm-cov coverage matrix row (apt.llvm.org bootstrap for
  clang >= 19 since 24.04 default repos cap at clang-18)
- conditional install of llvm-N vs gcovr based on coverage_tool
- artifact + codecov upload generalised to coverage.lcov | coverage.xml
- coverage cmake-args switched to *_FLAGS_DEBUG so the build action's
  stdlib flag isn't clobbered

temp (revert before merging to dev):
- matrix narrowed to just the llvm-cov row on non-main refs so we can
  iterate without burning runners
- 'coverage-llm' added to push trigger for direct-push CI runs
prior guard checked base_ref against ['dev','candidate','release'] which
excluded the very PR we're iterating on. drop the guard - this branch
is for iteration only and must be reverted before merging anyway.
grpc 1.50.1 hits -Werror=missing-template-arg-list-after-template-kw on
clang-19+. macOS clang already had this workaround in the conan profile;
mirror it to the Linux branch, gated on compiler==clang.
renames every non-nix workflow to .yml.disabled so they stop firing on
PR pushes while we iterate on the llvm-cov coverage row. MUST be
reverted before merging to dev.
bare 'rippled' wasn't on PATH and the build dir isn't '.', so the
profile-collection step failed with 'No such file or directory'.
splice the resolved absolute path back into Cov_EXECUTABLE before
the run command consumes it.
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 30, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.40%. Comparing base (5e8d26f) to head (273273d).
⚠️ Report is 1 commits behind head on dev.

Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #740      +/-   ##
==========================================
- Coverage   66.55%   66.40%   -0.16%     
==========================================
  Files         831      853      +22     
  Lines       78166    85020    +6854     
  Branches    44374    21321   -23053     
==========================================
+ Hits        52023    56457    +4434     
- Misses      17797    19820    +2023     
- Partials     8346     8743     +397     
Flag Coverage Δ
llvm 66.40% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

sublimator added 11 commits May 5, 2026 08:57
…deps_cxxflags

new dependencies action input `conan_deps_cxxflags` (json list, default
'[]') drives `tools.build:cxxflags` in the conan profile. clearly named
to indicate the flags only apply to conan dependency builds, not the
rippled build itself.

removes the per-os/per-compiler hardcoded workaround (linux clang
conditional + unconditional macOS block) that used to set the same flag
in two places kept-in-sync by hand.

call sites:
- xahau-ga-nix.yml: clang-20 coverage row gets the
  -Wno-missing-template-arg-list-after-template-kw workaround for grpc 1.50.1
- xahau-ga-macos.yml.disabled: same flag plumbed through (preserves
  prior behaviour when re-enabled)

drop the workaround entries when grpc is bumped past the fix.
two bugs caught in review:

- RippledSettings.cmake: the clang-only guard for coverage_tool=llvm ran
  before coverage_test could auto-enable coverage. so
  '-Dcoverage_tool=llvm -Dcoverage_test=Foo' on a gcc build slipped past
  the guard and produced an instrumentation/tool mismatch. move the
  guard after the auto-enable block.

- CodeCoverageLLVM.cmake: _find_llvm_cov_tools unconditionally
  preferred xcrun's tools on APPLE, which on a homebrew clang-N build
  would pair the user's clang with xcode's llvm-cov - exactly the
  version mismatch the helper is trying to avoid. gate the xcrun branch
  on CMAKE_CXX_COMPILER_ID == AppleClang.
- CodeCoverageLLVM.cmake: BASE_DIRECTORY was parsed but never used
  (no analog to gcovr's -r in the llvm-cov commands we emit). dead arg.
- xahau-ga-macos.yml.disabled: revert the conan_deps_cxxflags addition.
  the file is disabled so the edit was bit-rotting in unreachable code.
  whoever revives the macOS workflow can wire up the field then.
restore the conan_deps_cxxflags addition to xahau-ga-macos.yml.disabled.
prior commit framed it as dead code, but the .disabled rename is itself
a TEMP measure being reverted before merge - the field is needed for the
macOS workflow to keep building grpc 1.50.1 once it's re-enabled, since
the unconditional workaround was removed from the dependencies action.
reverts part of f8a30c5 - removing this line dropped the apple-clang
grpc workaround that the dependencies action used to apply
unconditionally before the matrix-driven refactor. the .disabled rename
is itself a TEMP measure being reverted before merge, so the field needs
to be in place when the macOS workflow comes back.
prior turn-of-events created two identical conan_deps_cxxflags entries
in the same matrix row from overlapping restore commits. keep one.
…-cov as sole codecov reporter

- restore the 8 .yml.disabled workflows back to .yml
- drop the gcc-13 gcov coverage matrix row; native clang-20 llvm-cov is
  now the only codecov reporter
- update minimal_matrix indices to match
- remove the TEMP narrowing block that filtered to just the llvm-cov row
- log line says '7 configs (build x6 + clang-20 llvm-cov coverage)' to
  reflect the new shape
prior shape (bare json list -> 'tools.build:cxxflags=[...]') leaked into
the consumer/rippled build via the conan-generated toolchain
(CMAKE_CXX_FLAGS_INIT). on macOS, where the build action doesn't
override CMAKE_CXX_FLAGS, this would silently apply the workaround flag
to rippled itself.

reshape the input to a json object keyed by Conan package pattern:
  {"grpc/*":["-Wno-..."]}

action emits package-pattern scoped lines:
  grpc/*:tools.build:cxxflags=["-Wno-..."]

flags only apply while building the matching dependency. consumer
toolchain stays clean. python validator rejects the consumer pattern
('&') and malformed shapes.
@sublimator sublimator marked this pull request as ready for review May 5, 2026 03:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant