Fix Runner loading and refactor code #977
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
elliotgunton
added
semver:patch
elliotgunton
force-pushed
the
runner-refactor
branch
from
ac8fab7
to
3c682b6
Compare
4 tasks
* Create a `_runner` submodule, where `util.py` lives * Maintains the hera.workflows.runner entrypoint Signed-off-by: Elliot Gunton <[email protected]>
* Isolating auto gen code changes, can ignore commit Signed-off-by: Elliot Gunton <[email protected]>
Signed-off-by: Elliot Gunton <[email protected]>
* Function is now testable, doesn't use global kwargs * Returns a value directly Signed-off-by: Elliot Gunton <[email protected]>
Signed-off-by: Elliot Gunton <[email protected]>
* Fixes value loading with a slightly hacky fix for now Signed-off-by: Elliot Gunton <[email protected]>
Signed-off-by: Elliot Gunton <[email protected]>
Signed-off-by: Elliot Gunton <[email protected]>
* Also tidy up types Signed-off-by: Elliot Gunton <[email protected]>
elliotgunton
force-pushed
the
runner-refactor
branch
from
3c682b6
to
b0a913e
Compare
elliotgunton
force-pushed
the
fix-runner-io-defaults
branch
from
e7a055d
to
cb8417c
Compare
flaviuvadan
reviewed
Mar 4, 2024
examples/workflows/upstream/title-and-descriptin-with-markdown.upstream.yaml
Outdated
Show resolved
Hide resolved
elliotgunton
force-pushed
the
runner-refactor
branch
2 times, most recently
from
907425a
to
c7c610c
Compare
Signed-off-by: Elliot Gunton <[email protected]>
elliotgunton
force-pushed
the
runner-refactor
branch
from
c7c610c
to
46e7098
Compare
flaviuvadan
approved these changes
Mar 4, 2024
There was a problem hiding this comment.
Just a relatively minor comment on "utils". Personally I am not a fan but it's also not a blocking thing. I approve bc this is not "critical" + these are private files for an experimental feature + gives us the chance to bring it up as a team and align on the convention we'd like to use (util vs specific modules vs something else)
examples/workflows/upstream/title-and-description-with-markdown.upstream.yaml
Show resolved
Hide resolved
| ) | ||
|
|
||
|
|
||
| def _runner(entrypoint: str, kwargs_list: List) -> Any: |
There was a problem hiding this comment.
This seems like a very central runner function. I wonder if it can go into something more "significant" than a util file. I don't want to pick too much on "util", to be clear, but organizing into an abstract util does not tell contributors much atm :(
There was a problem hiding this comment.
Yes I was also not 100% behind this, but I didn't want to have runner in the path again, which would end up with patch(..._runner.runner._runner) in tests.
Naming suggestions welcome!
* Script user guide was too long, split into main features. Fix internal links * Make pydantic io example into a runnable workflow - made it more obvious the scripts would need a custom image **Pull Request Checklist** - [x] Fixes #961 (fixes example) - [ ] Tests added - [x] Documentation/examples added - [x] [Good commit messages](https://cbea.ms/git-commit/) and/or PR title **Description of PR** Currently, the pydantic IO example does not work. Combined with fixes in PRs #974 and #977, this doc change shows how to use the Runner IO features. --------- Signed-off-by: Elliot Gunton <[email protected]>
elliotgunton
added a commit
that referenced
this pull request
Mar 4, 2024
**Pull Request Checklist** - [x] Fixes #962 - [x] Tests added - [ ] Documentation/examples added - [x] [Good commit messages](https://cbea.ms/git-commit/) and/or PR title **Description of PR** Primarily, this PR refactors the runner code. It also fixes the mapping and loading of kwargs to a RunnerInput object. Currently, the runner code is hard to follow. This PR refactors the functionality in the `runner.py` file into a `_runner` module with util submodules. The PR also makes the complex input mapping logic more testable. --------- Signed-off-by: Elliot Gunton <[email protected]>
elliotgunton
added a commit
that referenced
this pull request
Mar 4, 2024
**Pull Request Checklist** - [x] Fixes #962 - [x] Tests added - [ ] Documentation/examples added - [x] [Good commit messages](https://cbea.ms/git-commit/) and/or PR title **Description of PR** Currently, defaults are required when using Parameters in the RunnerInput. This PR allows defaults to be omitted. ## Changes from #977 - [x] Fixes #962 - [x] Tests added - [ ] Documentation/examples added - [x] [Good commit messages](https://cbea.ms/git-commit/) and/or PR title **Description of PR** Primarily, this PR refactors the runner code. It also fixes the mapping and loading of kwargs to a RunnerInput object. Currently, the runner code is hard to follow. This PR refactors the functionality in the `runner.py` file into a `_runner` module with util submodules. The PR also makes the complex input mapping logic more testable. ## Changes from #982 * Script user guide was too long, split into main features. Fix internal links * Make pydantic io example into a runnable workflow - made it more obvious the scripts would need a custom image **Pull Request Checklist** - [x] Fixes #961 (fixes example) - [ ] Tests added - [x] Documentation/examples added - [x] [Good commit messages](https://cbea.ms/git-commit/) and/or PR title **Description of PR** Currently, the pydantic IO example does not work. Combined with fixes in PRs #974 and #977, this doc change shows how to use the Runner IO features. --------- Signed-off-by: Elliot Gunton <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Checklist
Description of PR
Primarily, this PR refactors the runner code. It also fixes the mapping and loading of kwargs to a RunnerInput object.
Currently, the runner code is hard to follow. This PR refactors the functionality in the
runner.pyfile into a_runnermodule with util submodules. The PR also makes the complex input mapping logic more testable.