Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

output coverage summary (gitlab integration) #556

Open
gdesmott opened this issue Jan 8, 2021 · 11 comments · May be fixed by #1180
Open

output coverage summary (gitlab integration) #556

gdesmott opened this issue Jan 8, 2021 · 11 comments · May be fixed by #1180

Comments

@gdesmott
Copy link

gdesmott commented Jan 8, 2021

It would be great to have an easy way to output on stdout the coverage summary in a way easily parsable by gitlab.

@gdesmott
Copy link
Author

gdesmott commented Jan 8, 2021

Found a workaround using lcov:

- grcov . --binary-path ./target/debug/ -s . -t lcov --branch --ignore-not-existing --ignore "*target*"  -o lcov
- lcov -r lcov "/*" 2&> out
- grep lines out

And then I use \s*lines\.*:\s*([\d\.]+%) as regexp in gitlab as suggested here: https://www.greycastle.se/how-to-show-flutter-test-coverage-in-gitlab-ci/

@calixteman
Copy link
Collaborator

Any PR is welcome.
There are several examples of output so I'd say that isn't hard to do.

@gdesmott
Copy link
Author

gdesmott commented Jan 8, 2021

Would you add this as a new summary output type?

@calixteman
Copy link
Collaborator

I'm fine with summary.

@marco-c
Copy link
Collaborator

marco-c commented Jan 21, 2021

Duplicate of #468?

@gdesmott
Copy link
Author

No, #468 is about generating a full report in the Cobertura format, while this ticket is about just producing a summary (nb of lines covered / total nb of lines) for GitLab summary.

@detly
Copy link
Contributor

detly commented Feb 25, 2023

This is what I currently do for Gitlab. I use xmlstarlet, which is similar to jq but for XML data.

script:
    # Coverage commands go here...
    # Extract and print percentage from Cobertura. See Gitlab issue #21549
    - >
      export COVERAGE_DEC=$(
      xmlstarlet sel -t -v "number(//coverage/@line-rate)"
      <cobertura.xml 2>/dev/null
      )
    - export COVERAGE_PCT=$(echo "100 * $COVERAGE_DEC" | bc)
    - printf "gitlab-coverage %s%%\n" "$COVERAGE_PCT"
  coverage: '/^gitlab-coverage .+%$/'

Full CI file is here.

@edmundoferreira
Copy link
Contributor

edmundoferreira commented Mar 16, 2023

I think --summary is a nice approach. Since I faced the same issue I can maybe share here my current Gitlab CI/CD solution using grcov in a C++ codebase.

.ctest:
    stage: test
    script:
        - ctest --test-dir ./build -j $(nproc) --no-tests=error --output-on-failure --output-junit ctest_junit.xml
        - mkdir ./build/coverage/
        - grcov ./build -s ./ --threads $(nproc) --branch --output-types html,cobertura,markdown -o ./build/coverage/ --ignore '/usr/include*' --ignore '*build*' --ignore '*lib*'
        - cat ./build/coverage/markdown.md
    coverage: /Total coverage:.*\s+(\d*\.?\d+\%)/
    artifacts:
        paths:
            - ./build/coverage/
        reports:
            junit: build/ctest_junit.xml
            coverage_report:
                coverage_format: cobertura
                path: ./build/coverage/cobertura.xml

I output as markdown and cat it to the job shell.
Then use the following regex coverage: /Total coverage:.*\s+(\d*\.?\d+\%)/ to get the values into gitlab. This was my main driver for PR #981.
I use ctest to get junit test information.
This also has cobertura information displayed correctly in gitlab web code editor, shown on MR.

@hartwork
Copy link

hartwork commented Jun 5, 2023

@gdesmott thanks for the hint on lcov. I ended up using lcov --list <filename> for a summary, like this:

grcov_args=(
  --binary-path target/debug
  --source-dir src
  --ignore '/*'
  --branch
  # grcov needs help finding llvm-profdata, apparently
  --llvm-path "$(dirname "$(which llvm-profdata)")"
)
grcov "${grcov_args[@]}" -t html -o target/coverage/html .
grcov "${grcov_args[@]}" -t lcov -o target/coverage/lcov .
lcov --list target/coverage/lcov 2>/dev/null \
  | tee target/coverage/summary.txt

@busticated
Copy link

i assume --summary would also produce human-readable output. is that correct? i found my way here to file a feature request for that - it's pretty handy to have coverage info printed to the terminal without having to jump thru hoops and / or take on additional dependencies.

@marco-c
Copy link
Collaborator

marco-c commented Sep 26, 2023

Yes, I think that's the idea.

rshearman added a commit to rshearman/grcov that referenced this issue Apr 8, 2024
Add a --print-summary option along the lines of what gcovr supports
and outputting in a compatible format. The number of functions covered
isn't output (it is with gcovr) as this is more complicated to
calculate and it's unclear if it would be useful.

Fixes mozilla#556
@rshearman rshearman linked a pull request Apr 8, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants