✨ adding docker-api-proxy service ⚠️

[GitHK]

What do these changes do?

Why yet another service? It allows to use the docker API from a manager node.
Why is this important? Some services like the dynamic-scheduler require access to it in order to create and remove services and overlay networks (swarm scope network).

Bonuses

• added a way to combine multiple lifespan generators
• added shared single docker client tied to the app's lifecycle
• connected the dynamic-scheduler to the docker-api-proxy

Related issue/s

• closes ✨ adding endpoints to create and remove docker services to director-v2 #6993 since it's no longer required
• closes ✨adding endpoints to create and remove docker networks to director-v2 #6991 since it's no longer required
• closes add docker-api-proxy service to the stack that can allows to use the docker socket remotely form another service not running on a simcore node or potentially inside the cluster #7145
• dynamic scheduler: extract scheduling logic and move it away from director-v2 #5632
• Add configuration for new service docker-api-proxy osparc-ops-environments#965
• ♻️ Making docker-api-network internal #7222
• Add docker-api-proxy service osparc-ops-environments#969

How to test

Dev-ops checklist

• No ENV changes or I properly updated ENV (read the instruction)
•  merge this MR before merging this

[codecov]

Codecov Report

Attention: Patch coverage is 35.82090% with 43 lines in your changes missing coverage. Please review.

Project coverage is 87.04%. Comparing base (89d6628) to head (37b1f5d).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7070      +/-   ##
==========================================
- Coverage   87.06%   87.04%   -0.02%     
==========================================
  Files        1647     1642       -5     
  Lines       64308    64181     -127     
  Branches     1093     1188      +95     
==========================================
- Hits        55987    55865     -122     
+ Misses       8008     8003       -5     
  Partials      313      313              

Flag	Coverage Δ
integrationtests	65.29% <ø> (-0.03%)	⬇️
unittests	86.03% <35.82%> (-0.01%)	⬇️

Components	Coverage Δ
api	∅ <ø> (∅)
pkg_aws_library	94.02% <ø> (ø)
pkg_dask_task_models_library	97.09% <ø> (ø)
pkg_models_library	91.53% <100.00%> (+<0.01%)	⬆️
pkg_notifications_library	84.57% <ø> (ø)
pkg_postgres_database	88.25% <ø> (ø)
pkg_service_integration	70.03% <ø> (ø)
pkg_service_library	72.48% <0.00%> (-0.55%)	⬇️
pkg_settings_library	90.61% <85.71%> (-0.10%)	⬇️
pkg_simcore_sdk	85.46% <ø> (ø)
agent	96.46% <ø> (ø)
api_server	90.55% <ø> (ø)
autoscaling	96.08% <ø> (ø)
catalog	91.71% <ø> (ø)
clusters_keeper	99.24% <ø> (ø)
dask_sidecar	91.25% <ø> (ø)
datcore_adapter	93.19% <ø> (ø)
director	76.59% <ø> (ø)
director_v2	91.30% <ø> (+0.01%)	⬆️
dynamic_scheduler	97.19% <100.00%> (+0.01%)	⬆️
dynamic_sidecar	89.77% <ø> (ø)
efs_guardian	90.25% <ø> (ø)
invitations	93.28% <ø> (ø)
osparc_gateway_server	∅ <ø> (∅)
payments	92.66% <ø> (ø)
resource_usage_tracker	89.06% <ø> (+0.10%)	⬆️
storage	86.22% <ø> (ø)
webclient	∅ <ø> (∅)
webserver	84.79% <ø> (+<0.01%)	⬆️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 89d6628...37b1f5d. Read the comment docs.

[sanderegg]

some early comments

[sanderegg]

much better thanks!
There are a few things to still fix/think about:

• dockerfile in root mode
• who closes the aiohttp client?

[YuryHrytsuk]

Thanks for your work!

Let's discuss security implications before going ahead.

[YuryHrytsuk]

Thanks!

@GitHK please create a ticket to add this service to ops repository and assign to me

[sanderegg]

Thanks!, a few last comments

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[GitHK]

@Mergifyio queue

[mergify]

queue

🛑 The pull request has been removed from the queue default

The pull request can't be updated.

You can take a look at Queue: Embarked in merge queue check runs for more details.

In case of a failure due to a flaky test, you should first retrigger the CI.
Then, re-embark the pull request into the merge queue by posting the comment
@mergifyio refresh on the pull request.

[mergify]

This pull request has been removed from the queue for the following reason: pull request branch update failed.

The pull request can't be updated

You should look at the reason for the failure and decide if the pull request needs to be fixed or if you want to requeue it.

If you want to requeue this pull request, you need to post a comment with the text: @mergifyio requeue