Add example `functions` section to api-server #7398

bisgaard-itis · 2025-03-19T16:22:40Z

What's the point of this PR?

As part of out metamodeling efforts we plan to develop the concept of a function in the api server. These functions could point to concepts we already have in the api-server (studies or solvers) or to new types of functions. This PR does nothing more than setup a few files and connect a functions section in the api-server with a new functions domain in the api-server so that @wvangeit can start implementing the thing. I.e. it is a skeleton to guide the development. I.e. the functions which are introduced in this PR will eventually go away as we start to merge the real functionality for the functions.

What do these changes do?

Add functions section in api-server. Expose a single ping endpoint as an example
Add functions domain with example files of how to setup the domain.
Connect ping endpoint with webserver via rpc to webserver.

Related issue/s

fixes Add example functionsapi with a single endpoint which is connected all the way to the webserver via rpc #7348

How to test

Dev-ops checklist

No ENV changes or I properly updated ENV (read the instruction)

bisgaard-itis · 2025-03-19T16:25:52Z

@wvangeit please let me know if anything in this example is unclear. I am very happy to extend it to make it more realistic or add some actual functionality if required.

codecov · 2025-03-19T16:29:35Z

Codecov Report

Attention: Patch coverage is 65.30612% with 17 lines in your changes missing coverage. Please review.

Project coverage is 85.14%. Comparing base (b5230f1) to head (5d277a9).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7398      +/-   ##
==========================================
- Coverage   87.31%   85.14%   -2.17%     
==========================================
  Files        1712     1348     -364     
  Lines       66429    56072   -10357     
  Branches     1125      578     -547     
==========================================
- Hits        58004    47745   -10259     
- Misses       8105     8142      +37     
+ Partials      320      185     -135

Flag	Coverage Δ
integrationtests	`65.30% <52.00%> (-0.01%)`	⬇️
unittests	`83.89% <65.30%> (-2.59%)`	⬇️

Components	Coverage Δ
api	`∅ <ø> (∅)`
pkg_aws_library	`∅ <ø> (∅)`
pkg_dask_task_models_library	`∅ <ø> (∅)`
pkg_models_library	`∅ <ø> (∅)`
pkg_notifications_library	`∅ <ø> (∅)`
pkg_postgres_database	`∅ <ø> (∅)`
pkg_service_integration	`∅ <ø> (∅)`
pkg_service_library	`72.10% <0.00%> (-0.16%)`	⬇️
pkg_settings_library	`∅ <ø> (∅)`
pkg_simcore_sdk	`85.46% <ø> (ø)`
agent	`96.46% <ø> (ø)`
api_server	`90.65% <83.33%> (-0.03%)`	⬇️
autoscaling	`96.08% <ø> (ø)`
catalog	`92.14% <ø> (ø)`
clusters_keeper	`99.24% <ø> (ø)`
dask_sidecar	`91.25% <ø> (ø)`
datcore_adapter	`98.11% <ø> (ø)`
director	`76.59% <ø> (-0.10%)`	⬇️
director_v2	`91.30% <ø> (ø)`
dynamic_scheduler	`97.33% <ø> (ø)`
dynamic_sidecar	`90.11% <ø> (ø)`
efs_guardian	`89.79% <ø> (ø)`
invitations	`93.28% <ø> (ø)`
payments	`92.66% <ø> (ø)`
resource_usage_tracker	`89.12% <ø> (ø)`
storage	`84.75% <ø> (+0.46%)`	⬆️
webclient	`∅ <ø> (∅)`
webserver	`80.36% <88.00%> (-5.49%)`	⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b5230f1...5d277a9. Read the comment docs.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

GitHK

I'm a bit lost. What would be te final goal of this functions?
I see no reference to that

sanderegg

I am also not clear what this "function" api is. all this seems very hard-coded in the end... I thought this was something where you send a "function" to be executed in there... but this looks like you have to implement each function precisely instead...

services/api-server/src/simcore_service_api_server/api/routes/functions.py

...ages/service-library/src/servicelib/rabbitmq/rpc_interfaces/webserver/functions/functions.py

services/web/server/src/simcore_service_webserver/functions/_controller_rpc.py

services/web/server/src/simcore_service_webserver/functions/_service.py

pcrespov · 2025-03-20T12:46:39Z

services/web/server/src/simcore_service_webserver/functions/plugin.py

+from . import _controller_rpc
+
+
+def setup_functions(app: web.Application):


setup docorator is missing here! See other setup to check this

@app_module_setup( __name__, ModuleCategory.ADDON, settings_name="WEBSERVER_FUNCTIONS", logger=_logger, ) def setup_functions(app: web.Application) -> bool: ...

Another thing that is missing is to create a settings field and mark is as DEV-FEATURE for this domain e.g.

class ApplicationSettings(BaseApplicationSettings): # other fields WEBSERVER_FUNCTIONS: Annotated[ bool, Field( json_schema_extra={"auto_default_from_env": True, _X_DEV_FEATURE_FLAG: True} ), ] = False

services/web/server/src/simcore_service_webserver/functions/_controller_rpc.py

services/api-server/src/simcore_service_api_server/api/routes/functions.py

pcrespov · 2025-03-20T12:54:00Z

services/api-server/src/simcore_service_api_server/api/root.py

@@ -44,6 +45,7 @@ def create_router(settings: ApplicationSettings):
    router.include_router(
        licensed_items.router, tags=["licensed-items"], prefix="/licensed-items"
    )
+    router.include_router(functions.router, tags=["functions"], prefix="/functions")


I would then also enable this if DEV_FEATURE is enabled.

The idea is that as soon as @wvangeit merges, it can be used in master

bisgaard-itis · 2025-03-20T13:05:55Z

I'm a bit lost. What would be te final goal of this functions? I see no reference to that

Yes, sorry. That's on me. I did not explain well in the header what exactly the goal of this PR is. I will update it

bisgaard-itis · 2025-03-20T13:41:34Z

I am also not clear what this "function" api is. all this seems very hard-coded in the end... I thought this was something where you send a "function" to be executed in there... but this looks like you have to implement each function precisely instead...

Yes, actually this PR just introduce some "dummy" functions so Werner can start implementing the actual api and has a pattern to follow. I have updated the header of this PR with some more info

sonarqubecloud · 2025-03-20T13:52:05Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

sanderegg

ok thanks for the explanation. I am still a bit unsure. but let's see.

GitHK

Sorry, but I have doubts and issues with this approach. Please let's have chat about it. See my reasons below.

I am not convinced at all by these changes/pattern.

I believe that the current approach will just cause harm in the long run.

I see the following issues/uncertainties:

I did not understand the concept of a function
no generic interface to register, launch and get the result of a function
a function could also be long running (long lived), the current pattern will eventually fail (example, if the webserver is restarted)
why was the webserver chosen as the owner of these functions?
Can you please provide a schema/diagarm of how the interaction with a functions will work?

bisgaard-itis added 3 commits March 19, 2025 16:40

add dummy functions section in webserver add rpc interface

e4ed2fa

introduce functions section in api server and expose ping endpoint

4b49f93

fix minor issues and manual testing

5b1b06b

bisgaard-itis changed the title ~~7348 add dummy functions api~~ Add example functions section to api-server Mar 19, 2025

bisgaard-itis changed the title ~~Add example functions section to api-server~~ 👽️ Add example functions section to api-server Mar 19, 2025

bisgaard-itis self-assigned this Mar 19, 2025

bisgaard-itis added a:apiserver api-server service Meta-modeling labels Mar 19, 2025

bisgaard-itis added this to the The Awakening milestone Mar 19, 2025

bisgaard-itis requested a review from wvangeit March 19, 2025 16:23

Merge branch 'master' into 7348-add-dummy-functions-api

f65dff1

bisgaard-itis marked this pull request as ready for review March 19, 2025 16:24

bisgaard-itis requested review from pcrespov, sanderegg, GitHK and matusdrobuliak66 as code owners March 19, 2025 16:24

add __init__ to functions domain

1a18bf2

bisgaard-itis added 2 commits March 19, 2025 17:37

added functions section in api

5004019

make pylint happy

a24b9e7

GitHK reviewed Mar 20, 2025

View reviewed changes

sanderegg reviewed Mar 20, 2025

View reviewed changes

pcrespov requested changes Mar 20, 2025

View reviewed changes

bisgaard-itis added 4 commits March 20, 2025 14:11

Merge branch 'master' into 7348-add-dummy-functions-api

e52bac8

@pcrespov remove ping from oas

394587e

@sanderegg assert input @pcrespov annotate ping function

cb107a0

fix relative imports

e29dd1b

bisgaard-itis changed the title ~~👽️ Add example functions section to api-server~~ Add example functions section to api-server Mar 20, 2025

regenerate oas

5d277a9

bisgaard-itis requested review from GitHK and sanderegg March 20, 2025 13:54

sanderegg approved these changes Mar 20, 2025

View reviewed changes

GitHK requested changes Mar 21, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add example `functions` section to api-server #7398

Add example `functions` section to api-server #7398

bisgaard-itis commented Mar 19, 2025 •

edited

Loading

bisgaard-itis commented Mar 19, 2025

codecov bot commented Mar 19, 2025 •

edited

Loading

GitHK left a comment

sanderegg left a comment

pcrespov Mar 20, 2025

pcrespov Mar 20, 2025

pcrespov Mar 20, 2025

bisgaard-itis commented Mar 20, 2025 •

edited

Loading

bisgaard-itis commented Mar 20, 2025

sonarqubecloud bot commented Mar 20, 2025

sanderegg left a comment

GitHK left a comment

		from . import _controller_rpc


		def setup_functions(app: web.Application):

Add example functions section to api-server #7398

Are you sure you want to change the base?

Add example functions section to api-server #7398

Conversation

bisgaard-itis commented Mar 19, 2025 • edited Loading

What's the point of this PR?

What do these changes do?

Related issue/s

How to test

Dev-ops checklist

bisgaard-itis commented Mar 19, 2025

codecov bot commented Mar 19, 2025 • edited Loading

Codecov Report

GitHK left a comment

Choose a reason for hiding this comment

sanderegg left a comment

Choose a reason for hiding this comment

pcrespov Mar 20, 2025

Choose a reason for hiding this comment

pcrespov Mar 20, 2025

Choose a reason for hiding this comment

pcrespov Mar 20, 2025

Choose a reason for hiding this comment

bisgaard-itis commented Mar 20, 2025 • edited Loading

bisgaard-itis commented Mar 20, 2025

sonarqubecloud bot commented Mar 20, 2025

Quality Gate passed

sanderegg left a comment

Choose a reason for hiding this comment

GitHK left a comment

Choose a reason for hiding this comment

Add example `functions` section to api-server #7398

Add example `functions` section to api-server #7398

bisgaard-itis commented Mar 19, 2025 •

edited

Loading

codecov bot commented Mar 19, 2025 •

edited

Loading

bisgaard-itis commented Mar 20, 2025 •

edited

Loading