feat(coverage): add automatic path fixing for CI/CD environments #2338

brynary · 2025-08-21T13:18:40Z

Summary

Implements automatic path fixing for code coverage reports in CI/CD environments
Adds DefaultPathFixer transformer that removes common CI/CD path prefixes
Automatically applies path fixing when manual options are not specified

Details

This PR introduces the DefaultPathFixer transformer to automatically handle path normalization for code coverage reports generated in various CI/CD environments. When users don't specify manual path fixing options (--path-replace or --path-prefix), the system will automatically remove common CI/CD path prefixes to ensure coverage paths match the repository structure.

Supported CI/CD Systems

The DefaultPathFixer recognizes and removes path prefixes from:

CircleCI (/home/circleci/)
GitHub Actions (/home/runner/work/, /Users/runner/work/)
Travis CI (/home/travis/build/)
Jenkins (/var/lib/jenkins/workspace/)
GitLab CI (/builds/)
Bitbucket Pipelines (/opt/atlassian/pipelines/agent/build/)
AWS CodeBuild (/codebuild/output/)
Azure Pipelines (/home/vsts/work/)

Changes

Added DefaultPathFixer transformer (qlty-coverage/src/transformer.rs)
- Implements automatic detection and removal of CI/CD path prefixes
- Includes comprehensive unit tests for all supported CI systems
Updated publish planner (qlty-coverage/src/publish/planner.rs)
- Applies DefaultPathFixer when no manual path options are provided
- Maintains backward compatibility with existing manual configuration
Updated transform planner (qlty-coverage/src/transform/planner.rs)
- Applies DefaultPathFixer when no manual path options are provided
- Ensures consistent behavior across both publish and transform commands

Test Plan

Unit tests for DefaultPathFixer with all supported CI path prefixes
Integration tests for publish command with DefaultPathFixer
Integration tests for transform command with DefaultPathFixer
Verified backward compatibility with manual path options
Manual testing in actual CI/CD environments
Documentation update for new automatic path fixing feature

🤖 Generated with Claude Code

Implement DefaultPathFixer transformer that automatically removes common CI/CD path prefixes from code coverage reports when manual path fixing options are not specified. This improves the user experience by eliminating the need for manual configuration in most standard CI environments. Supported CI/CD systems: - CircleCI (/home/circleci/) - GitHub Actions (/home/runner/work/, /Users/runner/work/) - Travis CI (/home/travis/build/) - Jenkins (/var/lib/jenkins/workspace/) - GitLab CI (/builds/) - Bitbucket Pipelines (/opt/atlassian/pipelines/agent/build/) - AWS CodeBuild (/codebuild/output/) - Azure Pipelines (/home/vsts/work/) The DefaultPathFixer is automatically applied in both publish and transform commands when users don't specify --path-replace or --path-prefix options, making code coverage integration seamless across different CI/CD platforms. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

github-actions · 2025-08-21T13:18:47Z

No issue mentions found. Please mention an issue in the pull request description.

Use GitHub automation to close the issue when a PR is merged

qltysh · 2025-08-21T13:21:22Z

Diff Coverage for ubuntu-latest: The code coverage on the diff in this pull request is 98.0%.

Total Coverage for ubuntu-latest: This PR will increase coverage by 0.15%.

File Coverage Changes

Path	File Coverage Δ	Indirect
qlty-cli/src/commands/coverage/publish.rs	0.3
qlty-coverage/src/ci/github.rs	-0.3	√
qlty-coverage/src/publish/planner.rs	1.3
qlty-coverage/src/publish/processor.rs	0.0
qlty-coverage/src/transform/planner.rs	83.5
qlty-coverage/src/transformer.rs	4.9

🛟 Help

Diff Coverage: Coverage for added or modified lines of code (excludes deleted files). Learn more.
Total Coverage: Coverage for the whole repository, calculated as the sum of all File Coverage. Learn more.
File Coverage: Covered Lines divided by Covered Lines plus Missed Lines. (Excludes non-executable lines including blank lines and comments.)
- Indirect Changes: Changes to File Coverage for files that were not modified in this PR. Learn more.

qltysh · 2025-08-21T13:21:55Z

Diff Coverage for macos-15: The code coverage on the diff in this pull request is 98.0%.

Total Coverage for macos-15: This PR will increase coverage by 0.16%.

File Coverage Changes

Path	File Coverage Δ	Indirect
qlty-cli/src/commands/coverage/publish.rs	0.3
qlty-coverage/src/ci/github.rs	-0.3	√
qlty-coverage/src/publish/planner.rs	1.3
qlty-coverage/src/publish/processor.rs	0.0
qlty-coverage/src/transform/planner.rs	83.5
qlty-coverage/src/transformer.rs	5.8

🛟 Help

Diff Coverage: Coverage for added or modified lines of code (excludes deleted files). Learn more.
Total Coverage: Coverage for the whole repository, calculated as the sum of all File Coverage. Learn more.
File Coverage: Covered Lines divided by Covered Lines plus Missed Lines. (Excludes non-executable lines including blank lines and comments.)
- Indirect Changes: Changes to File Coverage for files that were not modified in this PR. Learn more.

The pattern r'^/?home/[^/]+/src/([^/]+/){3}' was too generic and could match legitimate project paths. Removed to avoid false positives.

noahd1 · 2025-09-10T22:52:35Z

@brynary I did a risk assessment of this pull request and also some thoughts on possible enhancements. Take away is that this is probably good to go as is.

Risks

Risk: over-eager pattern matching which transforms paths that shouldn't get transformed

These are the patterns:

            r"^/?home/circleci/project/",
            r"^/?(home|Users)/runner/work/[^/]+/[^/]+/",
            r"^/?(home|Users)/travis/build/[^/]+/[^/]+/",
            r"^/?(home|Users)/jenkins/jobs/[^/]+/workspace/",
            r"^/?Users/distiller/[^/]+/",
            r"^/?(home|Users)/[^/]+/workspace/[^/]+/[^/]+/",

Of course, anything is possible, but these seem largely safe.

We could introduce a flag like --skip-default-transforms but I'd wait until this is actually a real world problem before introducing it.

Risk: doesn't work in practice (missed or incorrect patterns)

There are unit tests for the patterns which are good. The risk I guess is that these CI providers have different paths than the ones we've specified, and in which case, the behavior will be the same as today, which is that a strip prefix argument must be supplied. Seems ok.

Risk: isn't backwards compatible -- doesn't properly apply a customers add or strip prefix transformation

The tests in qlty-coverage/src/publish/planner.rs seem pretty good. There isn't an end to end test however. We could consider adding that.

Enhancements

Think this can largely wait, but was curious on your take:

Denote in debug output that paths are getting auto-transformed (bonus: which transformation was applied)
Print a deprecation warning if the strip prefix provided matches an established pattern

brynary · 2025-09-10T22:58:54Z

Thanks this is super helpful!

We can release on a weekend to limit any thundering herd of problems if we make a mistake.

One thing we should add is output like a line like this:

Auto-path fixing: Enabled

I don't think that has to be dynamic. In other words I think it's fine to render Enabled if we are attempting auto path fixing regardless of whether any paths are adjusted.

(Since path fixing is applied per path, the question of "was auto path fixing applied?" is not binary anyway. We could in theory print the counts but I don't think that's needed yet.)

noahd1 · 2025-09-10T23:06:50Z

(Since path fixing is applied per path, the question of "was auto path fixing applied?" is not binary anyway. We could in theory print the counts but I don't think that's needed yet.)

Depends on perspective I guess -- it's binary if any path fixing was applied. That said, it's certainly simpler to state that Auto-Path fixing is enabled.

brynary · 2025-09-11T15:58:08Z

@noahd1 we need to add a path fix for: github.com/SOMETHING/SOMETHING which is a form that I've been seeing lately. (This isn't even a system path on disk it's virtual but we can fix it)

brynary · 2025-09-11T15:58:42Z

Depends on perspective I guess -- it's binary if any path fixing was applied. That said, it's certainly simpler to state that Auto-Path fixing is enabled.

Yeah, better than just "enabled" would be "Enabled but not applied" vs. "Enabled and applied"

Copilot

Pull Request Overview

This PR implements automatic path fixing for code coverage reports in CI/CD environments by introducing a DefaultPathFixer transformer that removes common CI/CD path prefixes when manual path fixing options are not specified.

Added DefaultPathFixer transformer with support for 8+ CI/CD systems (CircleCI, GitHub Actions, Travis CI, etc.)
Updated both publish and transform planners to automatically apply path fixing when manual options are absent
Added comprehensive test coverage for the new functionality and maintained backward compatibility

Reviewed Changes

Copilot reviewed 7 out of 15 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
qlty-coverage/src/transformer.rs	Implements DefaultPathFixer transformer with regex patterns for CI/CD path detection and Windows path normalization
qlty-coverage/src/transform/planner.rs	Conditionally applies DefaultPathFixer to transform operations when manual path options aren't provided
qlty-coverage/src/publish/planner.rs	Conditionally applies DefaultPathFixer to publish operations and tracks auto-fixing status
qlty-coverage/src/publish/plan.rs	Adds auto_path_fixing_enabled field to track when automatic path fixing is active
qlty-coverage/src/publish/processor.rs	Passes auto_path_fixing_enabled flag from plan to report
qlty-coverage/src/publish/report.rs	Adds auto_path_fixing_enabled field to Report struct
qlty-coverage/src/validate.rs	Updates test helper to include auto_path_fixing_enabled field

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-11T23:54:40Z

qlty-coverage/src/transformer.rs

+        let pattern_strings = vec![
+            r"^/?home/circleci/project/",
+            r"^/?(home|Users)/runner/work/[^/]+/[^/]+/",
+            r"^/?github.com/[^/]+/[^/]+/",
+            r"^/?(home|Users)/travis/build/[^/]+/[^/]+/",
+            r"^/?(home|Users)/jenkins/jobs/[^/]+/workspace/",
+            r"^/?Users/distiller/[^/]+/",


The regex patterns lack documentation explaining which CI/CD systems they target. Consider adding comments like // CircleCI, // GitHub Actions, etc. to clarify which pattern matches which CI system.

Suggested change

let pattern_strings = vec![

r"^/?home/circleci/project/",

r"^/?(home|Users)/runner/work/[^/]+/[^/]+/",

r"^/?github.com/[^/]+/[^/]+/",

r"^/?(home|Users)/travis/build/[^/]+/[^/]+/",

r"^/?(home|Users)/jenkins/jobs/[^/]+/workspace/",

r"^/?Users/distiller/[^/]+/",

let pattern_strings = vec![

// CircleCI

r"^/?home/circleci/project/",

// GitHub Actions

r"^/?(home|Users)/runner/work/[^/]+/[^/]+/",

// GitHub Actions (checkout path)

r"^/?github.com/[^/]+/[^/]+/",

// Travis CI

r"^/?(home|Users)/travis/build/[^/]+/[^/]+/",

// Jenkins

r"^/?(home|Users)/jenkins/jobs/[^/]+/workspace/",

// CircleCI (macOS)

r"^/?Users/distiller/[^/]+/",

// Azure Pipelines

Copilot · 2025-09-11T23:54:40Z

qlty-coverage/src/transformer.rs

+            return stripped.to_string();
+        }
+        // Handle any single letter drive
+        if path.len() >= 3 && path.chars().nth(1) == Some(':') && path.chars().nth(2) == Some('/') {


Calling path.chars().nth() multiple times is inefficient as it iterates through the string each time. Consider using path.chars().take(3).collect::<Vec<_>>() or checking bytes directly if ASCII is sufficient.

Suggested change

if path.len() >= 3 && path.chars().nth(1) == Some(':') && path.chars().nth(2) == Some('/') {

if path.as_bytes().get(1) == Some(&b':') && path.as_bytes().get(2) == Some(&b'/') {

Copilot · 2025-09-11T23:54:41Z

qlty-coverage/src/transformer.rs

+        if let Some(stripped) = path.strip_prefix("C:/") {
+            return stripped.to_string();
+        }
+        if let Some(stripped) = path.strip_prefix("D:/") {
+            return stripped.to_string();
+        }


The hardcoded drive letter checks for C:/ and D:/ are redundant since the generic check on lines 247-248 handles any single letter drive. Remove the specific C:/ and D:/ checks to eliminate duplication.

Suggested change

if let Some(stripped) = path.strip_prefix("C:/") {

return stripped.to_string();

}

if let Some(stripped) = path.strip_prefix("D:/") {

return stripped.to_string();

}

Copilot · 2025-09-11T23:54:41Z

qlty-coverage/src/publish/planner.rs

+        let auto_path_fixing_enabled = transformers
+            .iter()
+            .any(|t| format!("{:?}", t).contains("DefaultPathFixer"));


Using format!("{:?}", t).contains("DefaultPathFixer") to detect transformer type is fragile and depends on debug formatting. Consider adding a trait method like transformer_type() or using std::any::TypeId for more reliable type checking.

Copilot · 2025-09-11T23:54:41Z

qlty-coverage/src/transform/planner.rs

+        let has_default_fixer = transformers
+            .iter()
+            .any(|t| format!("{:?}", t).contains("DefaultPathFixer"));


Using format!("{:?}", t).contains("DefaultPathFixer") to detect transformer type is fragile and depends on debug formatting. Consider adding a trait method like transformer_type() or using std::any::TypeId for more reliable type checking.

brynary and others added 2 commits August 21, 2025 09:26

fix: remove generic src pattern from default path fixer

ede14ae

The pattern r'^/?home/[^/]+/src/([^/]+/){3}' was too generic and could match legitimate project paths. Removed to avoid false positives.

Merge branch 'main' into feat-default-path-fixer-coverage

3131426

noahd1 added 2 commits September 10, 2025 16:36

Print auto-path fixing enabled

8c8323e

Merge branch 'main' into feat-default-path-fixer-coverage

e2f8942

noahd1 added 3 commits September 11, 2025 16:35

Merge branch 'main' into feat-default-path-fixer-coverage

2728e30

Fix tests, move where info is output to

c9b0945

Add github.com/workspace/project pattern

04cbde8

noahd1 marked this pull request as ready for review September 11, 2025 23:53

Copilot AI review requested due to automatic review settings September 11, 2025 23:53

Copilot AI reviewed Sep 11, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(coverage): add automatic path fixing for CI/CD environments #2338

feat(coverage): add automatic path fixing for CI/CD environments #2338

Uh oh!

brynary commented Aug 21, 2025

Uh oh!

github-actions bot commented Aug 21, 2025

Uh oh!

qltysh bot commented Aug 21, 2025 •

edited

Loading

Uh oh!

qltysh bot commented Aug 21, 2025 •

edited

Loading

Uh oh!

noahd1 commented Sep 10, 2025 •

edited

Loading

Uh oh!

brynary commented Sep 10, 2025

Uh oh!

noahd1 commented Sep 10, 2025

Uh oh!

brynary commented Sep 11, 2025

Uh oh!

brynary commented Sep 11, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Sep 11, 2025

Uh oh!

Copilot AI Sep 11, 2025

Uh oh!

Copilot AI Sep 11, 2025

Uh oh!

Copilot AI Sep 11, 2025

Uh oh!

Copilot AI Sep 11, 2025

Uh oh!

Uh oh!

	if path.len() >= 3 && path.chars().nth(1) == Some(':') && path.chars().nth(2) == Some('/') {
	if path.as_bytes().get(1) == Some(&b':') && path.as_bytes().get(2) == Some(&b'/') {

feat(coverage): add automatic path fixing for CI/CD environments #2338

Are you sure you want to change the base?

feat(coverage): add automatic path fixing for CI/CD environments #2338

Uh oh!

Conversation

brynary commented Aug 21, 2025

Summary

Details

Supported CI/CD Systems

Changes

Test Plan

Uh oh!

github-actions bot commented Aug 21, 2025

Uh oh!

qltysh bot commented Aug 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qltysh bot commented Aug 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

noahd1 commented Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Risks

Risk: over-eager pattern matching which transforms paths that shouldn't get transformed

Risk: doesn't work in practice (missed or incorrect patterns)

Risk: isn't backwards compatible -- doesn't properly apply a customers add or strip prefix transformation

Enhancements

Uh oh!

brynary commented Sep 10, 2025

Uh oh!

noahd1 commented Sep 10, 2025

Uh oh!

brynary commented Sep 11, 2025

Uh oh!

brynary commented Sep 11, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

qltysh bot commented Aug 21, 2025 •

edited

Loading

qltysh bot commented Aug 21, 2025 •

edited

Loading

noahd1 commented Sep 10, 2025 •

edited

Loading