Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate and push report #17

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

FriedrichRober
Copy link

@FriedrichRober FriedrichRober commented Feb 25, 2022

❗ Before merge, we need to create a data and gh-pages branch. ❗

Current Procedure:

  • create an artifact for each job in the test matrix that contains the result of the job (txt file)
  • generate a report (md file) that compares these results against latest, which is a symlink to the latest report directory
  • update latest symlink
  • generate badge (json file) that is used in README.md (via creating a request to shields.io, a free service that generates and provides this badge)
  • upload this report to data branch under hash
  • upload new redirect (html file) to gh-pages that is used in README.md for referencing the latest report (it seems to me that symlinks cannot be redirected natively in github markdown nor github pages; i decided it is better to update the gh-pages branch instead of the README.md on the main branch each time a report is created.)

Missing Features:

  • ⚠️ gap_build_version in the docker does not contain the hash to the commit when it was built
  • the workflow pkg-tests.yml uses standard and minimal setup, but we only store the state of minimal setup
    • Suggestion: store a json file instead of a txt file that only contains the result of the job. This file could also contain metadata about the package, like version and link to github)
  • use docker information for saving multiple reports and not just one
  • activate reports for all tests and not just pkg-tests.yml
  • generation of reports based on trigger via context github.event_name:
    • data folder structure
      • daily contains daily reports from schedule trigger
      • hash contains reports from pull request trigger or a comment trigger as in Julia
      • manual contains reports from workflow dispatch trigger
      • latest symlink to latest report in daily
    • for hash compare against latest hash in history that is contained in hash
    • for others compare against latest
  • references to FriedrichRober/integration should be removed.

Copy link
Contributor

@wilfwilson wilfwilson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of comments for now (I haven't looked in detail yet). I have pushed empty branches gh-pages and data to this repo.

The references to FriedrichRober/integration should probably be removed.

report:
name: "Report"
needs: individual-package
if: always()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, at least once we have finished testing, we only want to generate reports when we have run this workflow with all packages.

Suggested change
if: always()
if: ${{ always() && github.event.inputs.packages == '' }}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also save those in a different location, for example the suggested manual folder that could contain all reports created by a workflow dispatch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants