Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adjust results location resolution to pass references between Workflow steps #360

Merged
merged 16 commits into from
Nov 18, 2021

Conversation

fmigneault
Copy link
Collaborator

@fmigneault fmigneault commented Nov 11, 2021

Changes:

  • Add map_wps_output_location utility function to handle recurrent mapping of weaver.wps_output_dir back and
    forth with resolved weaver.wps_output_url.
  • Add more detection of map-able WPS output location to avoid fetching files unnecessarily. Common cases
    are Workflow running multiple steps on the same server or Application Package Process that reuses an output
    produced by a previous execution. Relates to [Feature] Explicit 'fetch' parameter for some given input #183.

Fixes:

ToDo

…eps during Workflow execution (fixes #358) + remove some useless file fetch steps (relates to #183)
@github-actions github-actions bot added ci/doc Issue related to documentation of the package feature/CWL Issue related to CWL support labels Nov 11, 2021
@fmigneault fmigneault added the process/workflow Related to a Workflow process. label Nov 11, 2021
@fmigneault fmigneault self-assigned this Nov 11, 2021
dbyrns
dbyrns previously approved these changes Nov 11, 2021
Copy link
Contributor

@dbyrns dbyrns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job!

@github-actions github-actions bot added ci/operations Related to CI operations (actions, execution, install, builds, etc.) ci/tests Tests of the package and features labels Nov 12, 2021
@github-actions github-actions bot added feature/job Issues related to job execution, reporting and logging. process/builtin Issue related to builtin application processes process/wps3 Issue related to WPS 3.x (REST-JSON) processes support labels Nov 16, 2021
@codecov
Copy link

codecov bot commented Nov 18, 2021

Codecov Report

Merging #360 (0d9d2e8) into master (eccd2ed) will increase coverage by 0.50%.
The diff coverage is 76.82%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #360      +/-   ##
==========================================
+ Coverage   78.24%   78.74%   +0.50%     
==========================================
  Files          63       63              
  Lines       10406    10469      +63     
  Branches     1564     1582      +18     
==========================================
+ Hits         8142     8244     +102     
+ Misses       1773     1726      -47     
- Partials      491      499       +8     
Impacted Files Coverage Δ
weaver/typedefs.py 0.00% <ø> (ø)
weaver/processes/sources.py 46.29% <25.00%> (+1.85%) ⬆️
weaver/processes/builtin/__init__.py 72.27% <33.33%> (-1.00%) ⬇️
weaver/wps/utils.py 74.12% <58.82%> (-6.45%) ⬇️
weaver/processes/wps_workflow.py 50.79% <66.66%> (+1.27%) ⬆️
weaver/utils.py 81.56% <76.92%> (+0.20%) ⬆️
weaver/processes/wps3_process.py 57.39% <79.22%> (+7.13%) ⬆️
weaver/processes/wps_package.py 77.82% <82.35%> (+5.92%) ⬆️
weaver/config.py 77.55% <100.00%> (ø)
weaver/processes/execution.py 82.05% <100.00%> (+0.05%) ⬆️
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eccd2ed...0d9d2e8. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/doc Issue related to documentation of the package ci/operations Related to CI operations (actions, execution, install, builds, etc.) ci/tests Tests of the package and features feature/CWL Issue related to CWL support feature/job Issues related to job execution, reporting and logging. process/builtin Issue related to builtin application processes process/workflow Related to a Workflow process. process/wps3 Issue related to WPS 3.x (REST-JSON) processes support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Missing parameter value in workflow
2 participants