Full circle reusing payloads (alpha quality) reached! #100
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When a WRROC is generated for a CWL workflow execution, two workflows are registered in it: the original one and the consolidated one. The original one is used to generated the consolidated one, and the consolidated one is used to run the workflow. But the containers were not properly associated to the executions of this last workflow.
* Added ContainerCacheHandler, where all the needed methods to manage container image snapshots are included. * Simplified several parameters passed and used to instantiate ContainerFactory and its descendants. * Rewritten SingularityContainerFactory, so it uses new instance of ContainerCacheHandler.
* Rewritten DockerContainerFactory, so it uses new instance of ContainerCacheHandler. * Added an additional helper method, to know which files to expect in the staged working directories.
* Rewritten PodmanContainerFactory, so it uses new instance of ContainerCacheHandler.
…tyContainerFactory.
…d, in order to support injected container images in the near commits.
…ustom docker registries
… of a label are properly parsed. Also, remove duplicated declaration, add an abstract declaration for local image manifests and add an additional check of the image digest.
…bel, and no metadata could be fetched back.
… is really a Container one
…d container image is more correct. Also, added initial Singularity container factory tests
…includes the registry. Also, shared method with other container tests has been moved to `tests.containers.util`
… of `docker inspect`, to avoid possible collisions with other inspectable objects.
Also, fixed several misbehaviours and inconsistencies among them.
…he strictness. Also, added all the needed intermediate method parameters to propagate the values of these new parameters to the WF constructor.
…low, container images, cached inputs and environment.
This is needed to properly use zipfile.Path in several classes.
…yload is in a zip archive
…uts and environment
* First, both input and environment SPARLQ queries related to executions and staged workflows have been rebuilt, to improve them and give the row elements the very same column names. * Then, a common method which performs input file validation has been developed, in order to discard "smelly" content. * Last, the MaterializedInput instances are properly built by the common method.
…of "glorified strings"
…ntainer factories
…tead of bare strings
Also, minor adjustments in method type annotations.
…ontent and LocalWorkflow. This transition has caused major code changes in path handling, and also indirectly in workflow engines and container factories.
Invalidation of cached inputs and environment for replicability scenarios where either a cached input or environment is replaced with a new, unmaterialized one, has been implemented. Also, rescued operational containers for re-staging scenario.
* Licences always have to be URIs, to properly validate against the staging definition schema. * Staging definition params were wrongly labelled in their `c-l-a-s-s`. * Also, licence curation method is now a public one at wfexs_backend.WfExSBackend instance.
Also, remove duplicated code moved to wfexs_backend in a previous commit.
…nces also work as source.
… not behave as expected
…etc... PS: The code still fails in some points
…ouple of places.
…w working with a simple CWL workflow!!!
Next releases will be -alpha, -beta, -rc and 1.0.0.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.