Add Python test coverage by greenc-FNAL · Pull Request #250 · Framework-R-D/phlex

greenc-FNAL · 2026-01-13T17:11:03Z

maintenance: Align coverage.sh with CI workflow
Add Python test execution and coverage monitoring
Improve Python coverage target with proper environment variables
Update documentation for Python coverage support
Address code review feedback
Improve Python executable detection in coverage.sh
Standardize Python package discovery across project

greenc-FNAL · 2026-01-13T17:12:19Z

@phlexbot format

github-actions · 2026-01-13T17:12:44Z

Automatic cmake-format fixes pushed (commit 99a85da).
⚠️ Note: Some issues may require manual review and fixing.

github-actions · 2026-01-13T17:12:46Z

No automatic clang-format fixes were necessary.

codecov · 2026-01-13T17:31:04Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

❌ Your project status has failed because the head coverage (75.63%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.

@@            Coverage Diff             @@
##             main     #250      +/-   ##
==========================================
+ Coverage   75.43%   75.63%   +0.20%     
==========================================
  Files         124      124              
  Lines        2902     2902              
  Branches      507      507              
==========================================
+ Hits         2189     2195       +6     
+ Misses        492      486       -6     
  Partials      221      221

Flag	Coverage Δ
unittests	`75.63% <ø> (+0.20%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 4 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d3cc3ed...333ba47. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

This pull request adds Python test coverage monitoring to the Phlex project, integrating pytest-cov for Python code coverage alongside existing C++ coverage infrastructure. The changes standardize Python package discovery across the project, switching from Python3 to Python for CMake find_package calls, and update the default coverage preset from GCC to Clang to better align with the CI workflow.

Changes:

Added Python coverage support via pytest-cov with automatic report generation in XML and HTML formats
Standardized Python package discovery using find_package(Python 3.12) across CMakeLists.txt and CreateCoverageTargets.cmake
Changed default coverage preset from coverage-gcc to coverage-clang to mirror CI workflow
Added new python command to coverage.sh for standalone Python coverage generation
Integrated Python coverage into GitHub Actions CI workflow with non-fatal reporting

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
test/python/CMakeLists.txt	Added pytest-cov module check and conditional PYTEST_COMMAND configuration with coverage flags when ENABLE_COVERAGE is ON
scripts/coverage.sh	Added generate_python_coverage function, changed default preset to coverage-clang, updated usage documentation, improved preset handling and command-line parsing
scripts/README.md	Documented new python command, updated command descriptions to indicate C++ and Python coverage support, added Python coverage requirements section
Modules/private/CreateCoverageTargets.cmake	Standardized to `find_package(Python 3.12)`, updated all Python_EXECUTABLE references, added coverage-python target, updated coverage-clean to remove Python artifacts
.github/workflows/coverage.yaml	Added Python coverage report generation step with graceful handling when pytest-cov unavailable, integrated Python coverage artifacts into upload process

scripts/coverage.sh

We use the module, so check for the module—availability of the command is irrelevant. Per Framework-R-D#250 (comment) Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Refactors the `coverage.sh` script to align its behavior, particularly for the Clang toolchain, with the CI coverage workflow. Key changes: - Sets the `LLVM_PROFILE_FILE` environment variable during the test phase for Clang to ensure correct collection of raw profile data, mirroring the CI setup. - Changes the default preset from `coverage-gcc` to `coverage-clang` to match the CI standard. - Adds HTML report generation for the Clang toolchain, bringing it to feature parity with the GCC preset within the script. - Simplifies the logic for the Clang path by removing irrelevant and incorrect GCC-specific staleness checks. - Improves the argument parsing to be more robust and reliable. Refactors the coverage.sh script to align its behavior with the CI workflow. - Implements support for uploading LLVM/Clang coverage reports to Codecov. - Improves the help text to clarify the differences between the GCC and Clang workflows. - Adds validation for the --preset flag. - Changes the exit code to 1 when no commands are provided. - Improves comments and file URL generation.

- Add pytest-cov integration to test/python/CMakeLists.txt - Update CreateCoverageTargets.cmake to add coverage-python target - Add python command to coverage.sh script - Update .github/workflows/coverage.yaml to collect Python coverage - Include Python coverage files in artifact bundles and Codecov uploads Co-authored-by: greenc-FNAL <2372949+greenc-FNAL@users.noreply.github.com>

- Add PHLEX_INSTALL environment variable to coverage-python target - Update documentation in CreateCoverageTargets.cmake header - Ensure Python coverage target has necessary environment setup Co-authored-by: greenc-FNAL <2372949+greenc-FNAL@users.noreply.github.com>

- Add python command to coverage.sh commands table - Document Python coverage requirements (pytest, pytest-cov) - Add example for Python-only coverage generation - Update command descriptions to note C++ and Python support Co-authored-by: greenc-FNAL <2372949+greenc-FNAL@users.noreply.github.com>

- Remove unused HAS_PYTEST variable from test/python/CMakeLists.txt - Remove unused PYTEST_EXECUTABLE find_program call - Use Python3_EXECUTABLE -m pytest consistently throughout Co-authored-by: greenc-FNAL <2372949+greenc-FNAL@users.noreply.github.com>

- Add fallback detection for both python3 and python commands - Use detected Python command for pytest-cov check - Make script more robust across different Python installations Co-authored-by: greenc-FNAL <2372949+greenc-FNAL@users.noreply.github.com>

- Change from find_package(Python3) to find_package(Python 3.12) - Replace all Python3_FOUND and Python3_EXECUTABLE references with Python_* - Ensure consistent Python interpreter detection across all CMake files - Aligns CreateCoverageTargets.cmake with test/python/CMakeLists.txt Co-authored-by: greenc-FNAL <2372949+greenc-FNAL@users.noreply.github.com>

We use the module, so check for the module—availability of the command is irrelevant. Per Framework-R-D#250 (comment) Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

greenc-FNAL · 2026-01-14T15:46:24Z

@phlexbot format

github-actions · 2026-01-14T15:46:49Z

Automatic cmake-format fixes pushed (commit 333ba47).
⚠️ Note: Some issues may require manual review and fixing.

github-actions · 2026-01-14T15:46:52Z

No automatic clang-format fixes were necessary.

We use the module, so check for the module—availability of the command is irrelevant. Per #250 (comment) Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

greenc-FNAL marked this pull request as ready for review January 13, 2026 19:57

Copilot AI review requested due to automatic review settings January 13, 2026 19:57

Copilot started reviewing on behalf of greenc-FNAL January 13, 2026 19:57 View session

Copilot AI reviewed Jan 13, 2026

View reviewed changes

scripts/coverage.sh Outdated Show resolved Hide resolved

google-labs-jules bot and others added 9 commits January 14, 2026 09:20

Address code review feedback

1b59d6f

- Remove unused HAS_PYTEST variable from test/python/CMakeLists.txt - Remove unused PYTEST_EXECUTABLE find_program call - Use Python3_EXECUTABLE -m pytest consistently throughout Co-authored-by: greenc-FNAL <2372949+greenc-FNAL@users.noreply.github.com>

Apply cmake-format fixes

34b1ec6

Check for pytest module instead of command

ca41803

We use the module, so check for the module—availability of the command is irrelevant. Per Framework-R-D#250 (comment) Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

greenc-FNAL force-pushed the copilot/add-python-test-coverage branch from 9d68111 to ca41803 Compare January 14, 2026 15:36

Apply cmake-format fixes

333ba47

knoepfel approved these changes Jan 14, 2026

View reviewed changes

greenc-FNAL merged commit 2cb061b into Framework-R-D:main Jan 14, 2026
35 of 36 checks passed

greenc-FNAL mentioned this pull request Jan 14, 2026

coverage.sh: improve consistency with coverage workflow #212

Closed

greenc-FNAL deleted the copilot/add-python-test-coverage branch January 20, 2026 23:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Python test coverage#250

Add Python test coverage#250
greenc-FNAL merged 10 commits intoFramework-R-D:mainfrom
greenc-FNAL:copilot/add-python-test-coverage

greenc-FNAL commented Jan 13, 2026

Uh oh!

greenc-FNAL commented Jan 13, 2026

Uh oh!

github-actions bot commented Jan 13, 2026

Uh oh!

github-actions bot commented Jan 13, 2026

Uh oh!

codecov bot commented Jan 13, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

greenc-FNAL commented Jan 14, 2026

Uh oh!

github-actions bot commented Jan 14, 2026

Uh oh!

github-actions bot commented Jan 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

greenc-FNAL commented Jan 13, 2026

Uh oh!

greenc-FNAL commented Jan 13, 2026

Uh oh!

github-actions bot commented Jan 13, 2026

Uh oh!

github-actions bot commented Jan 13, 2026

Uh oh!

codecov bot commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

greenc-FNAL commented Jan 14, 2026

Uh oh!

github-actions bot commented Jan 14, 2026

Uh oh!

github-actions bot commented Jan 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Jan 13, 2026 •

edited

Loading