ci: Test pathogen repo CI builds with the final packages #27
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1 task
|
Confirmed it's getting the right package. |
|
Two failing jobs seem to be issues with those pathogen repos? but they don't fail with the Docker runtime… so hmm. ncov fails in This was also recently reported by a user. So something's up here… Conda runtime is a common factor. seasonal-flu fails with |
1 task
[ Commit message based on that of 12000a20 in nextstrain/docker-base.¹ Code changes also based on that commit, plus subsequent commits.² ] A useful check for if new packages will break our pathogen builds. I included all pathogen repos that already use our pathogen-repo-ci reusable workflow. It should be minimal effort to maintain this list over time—I expect it to only grow—but perhaps in the future we will want to abstract it out into a shared list of known pathogen repos. I don't like that we have to copy the build-args for a few of the repos here since it'll be easy for this copy to diverge from the repo's authoritative build-args, but it's necessary for now. Over time as we work towards increased automation of pathogen builds, I think we can get rid of this build-args copy by further standardizing how each repo configures itself for automation. For example, instead of specifying build-args in a repo's CI workflow, the args for CI could be stored in a broader workflow metadata file (e.g. nextstrain-workflow.yaml) read by pathogen-repo-ci, or defined by some other convention. An alternative to directly running pathogen-repo-ci against each repo here would be instead triggering the CI workflows themselves within each repo. The downside to that is it would divorce the outcomes of those workflows from this one and render them not visible from PRs in this repo. It would also require updates to each repo to support triggering and passing in of additional parameters (i.e. for the package). And finally those CI workflows sometimes run other jobs, like linting and other integration tests (e.g. with Cram), that aren't always necessary to run with a new package. Related-to: <nextstrain/docker-base#148> Related-to: <nextstrain/docker-base#150> Related-to: <nextstrain/docker-base#151> Related-to: <nextstrain/docker-base#154> ¹ <nextstrain/docker-base@12000a20> ² <nextstrain/docker-base@bc22a0bc> <nextstrain/docker-base@0a20a474> <nextstrain/docker-base@75254e92>
tsibley
force-pushed
the
trs/test-pathogen-repo-ci
branch
from
5a1339a
to
abd21f3
Compare
|
Those failures should be investigated, but they shouldn't block merging this PR. |
victorlin
approved these changes
May 11, 2023
tsibley
force-pushed
the
trs/test-pathogen-repo-ci
branch
from
abd21f3
to
84ba40c
Compare
|
seasonal-flu issue was caused by Augur 22.0.0 change to |
|
Excellent work @tsibley! This is super helpful! The ncov failure is here: @huddlej your fix does resolve it, just reran the job and it fails only for ncov now, no longer seasonal-flu |
|
Aha, the reason |
This was referenced May 15, 2023
corneliusroemer
added a commit
to nextstrain/augur
that referenced
this pull request
May 15, 2023
Resolves #1215 Warn instead error when no nodes in a node data json, fixing issue introduced recently in PR #728 In PR #728, extra node data validation was introduced. In particular, files without information for either `nodes` or `branches` caused erroring. This is problematic for test scripts that may produce empty node data in test cases. This PR removes the eager validation. In the future we could reintroduce it as a warning. And possibly an error but with opt-out. This type of node data json was previously errored on by augur export, it is now accepted again: ```json { "nodes": {}, "rbd_level_details": {} } ``` <!-- Start typing the name of a related issue and GitHub will auto-suggest the issue number for you. --> Fixes the ncov pathogen-CI issue: nextstrain/conda-base#27 (comment) What steps should be taken to test the changes you've proposed? If you added or changed behavior in the codebase, did you update the tests, or do you need help with this? - [x] nextstrain/conda-base#27 (comment) is fixed, export now accepts empty nodes dicts again
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[ Commit message based on that of 12000a20 in nextstrain/docker-base.¹
Code changes also based on that commit, plus subsequent commits.² ]
A useful check for if new packages will break our pathogen builds.
I included all pathogen repos that already use our pathogen-repo-ci reusable workflow. It should be minimal effort to maintain this list over time—I expect it to only grow—but perhaps in the future we will want to abstract it out into a shared list of known pathogen repos.
I don't like that we have to copy the build-args for a few of the repos here since it'll be easy for this copy to diverge from the repo's authoritative build-args, but it's necessary for now. Over time as we work towards increased automation of pathogen builds, I think we can get rid of this build-args copy by further standardizing how each repo configures itself for automation. For example, instead of specifying build-args in a repo's CI workflow, the args for CI could be stored in a broader workflow metadata file (e.g. nextstrain-workflow.yaml) read by pathogen-repo-ci, or defined by some other convention.
An alternative to directly running pathogen-repo-ci against each repo here would be instead triggering the CI workflows themselves within each repo. The downside to that is it would divorce the outcomes of those workflows from this one and render them not visible from PRs in this repo. It would also require updates to each repo to support triggering and passing in of additional parameters (i.e. for the package). And finally those CI workflows sometimes run other jobs, like linting and other integration tests (e.g. with Cram), that aren't always necessary to run with a new package.
Related-to: nextstrain/docker-base#148
Related-to: nextstrain/docker-base#150
Related-to: nextstrain/docker-base#151
Related-to: nextstrain/docker-base#154
¹ nextstrain/docker-base@12000a20
² nextstrain/docker-base@bc22a0bc
nextstrain/docker-base@0a20a474
nextstrain/docker-base@75254e92
Testing