Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce links for async jobs actions (hateos style) in webserver #7320

Conversation

bisgaard-itis
Copy link
Contributor

@bisgaard-itis bisgaard-itis commented Mar 7, 2025

What do these changes do?

This PR redesigns both the webservers api for the async jobs (currently: computing the size of a path and zipping). It also improves on the design of the rpc interface (currently in storage, but to be moved to servicelib in the future).

Webserver task api

  • Overwrite long running tasks api and move the "legacy" implementation to /v0/tasks-legacy which is not included in the openapi specs. The new interface does not break backwards compatibility. The new GET /v0/tasks endpoint retrives both the legacy long running tasks as well as the new async jobs. The HATEOS links will point to the right endpoints for getting the status/results or abort endpoints.
  • Add links (get_status, abort, get_result) to response bodies for async job endpoints in the webserver.

Storage async jobs rpc interface

  • Raise exceptions in get_result method (for the async jobs in the storage worker) when an error occurred while running the job, it was aborted or is not yet done.

  • Raise exceptions if status/result/abort is requested but the job doesn't exist.

  • Improve unit testing both in webserver and storage related to async jobs.

Related issue/s

How to test

Dev-ops checklist

@bisgaard-itis bisgaard-itis self-assigned this Mar 7, 2025
@bisgaard-itis bisgaard-itis added the a:webserver issue related to the webserver service label Mar 7, 2025
@bisgaard-itis bisgaard-itis added this to the The Awakening milestone Mar 7, 2025
Copy link

codecov bot commented Mar 7, 2025

Codecov Report

Attention: Patch coverage is 85.12821% with 29 lines in your changes missing coverage. Please review.

Project coverage is 87.15%. Comparing base (fb2ccf3) to head (253f65f).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7320      +/-   ##
==========================================
- Coverage   87.30%   87.15%   -0.16%     
==========================================
  Files        1709     1546     -163     
  Lines       66361    62651    -3710     
  Branches     1125      891     -234     
==========================================
- Hits        57938    54601    -3337     
+ Misses       8103     7791     -312     
+ Partials      320      259      -61     
Flag Coverage Δ
integrationtests 65.32% <63.82%> (+0.08%) ⬆️
unittests 86.26% <85.12%> (-0.21%) ⬇️
Components Coverage Δ
api 76.84% <ø> (ø)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library 92.04% <100.00%> (+0.03%) ⬆️
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration 70.03% <ø> (ø)
pkg_service_library 72.26% <0.00%> (-0.07%) ⬇️
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 85.46% <ø> (ø)
agent 96.46% <ø> (ø)
api_server 90.68% <ø> (ø)
autoscaling 96.08% <ø> (ø)
catalog 92.14% <ø> (ø)
clusters_keeper 99.24% <ø> (ø)
dask_sidecar 91.25% <ø> (ø)
datcore_adapter 98.11% <ø> (ø)
director 76.68% <ø> (+0.09%) ⬆️
director_v2 91.30% <ø> (ø)
dynamic_scheduler 97.33% <ø> (ø)
dynamic_sidecar 90.11% <ø> (ø)
efs_guardian 89.79% <ø> (ø)
invitations 93.28% <ø> (ø)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.02% <ø> (-0.17%) ⬇️
storage 84.56% <96.55%> (+0.41%) ⬆️
webclient ∅ <ø> (∅)
webserver 85.84% <100.00%> (+0.03%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fb2ccf3...253f65f. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks ok with me, some minor things

@bisgaard-itis bisgaard-itis requested a review from odeimaiz March 19, 2025 14:00
Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is very nice thanks! I added a few comments. we can discuss if needed.

@bisgaard-itis bisgaard-itis enabled auto-merge (squash) March 20, 2025 11:58
@bisgaard-itis bisgaard-itis merged commit b5230f1 into ITISFoundation:master Mar 20, 2025
141 of 144 checks passed
@bisgaard-itis bisgaard-itis deleted the introduce-links-for-webserver-async-jobs branch March 20, 2025 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:webserver issue related to the webserver service
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants