Add support of using custom env variables #1448

AnastasiaSliusar · 2024-08-01T12:21:36Z

References

This PR is the part of solution of the feature where users can set up custom env variables for a kernel which has not been run yet.
UI is on AnastasiaSliusar/jupyterlab#2

Code changes

This PR includes accept_kernel_env_var flag which can be used during launching the application. If it is true then a user can see UI for configuring custom env variables when they select another not used kernel from the kernel select dialog or when they call a context menu for a certain kernel icon on Launcher
This PR includes support of custom_env_vars variable which has user selection and it is used to setup custom env variables for a certain kernel

Zsailer · 2024-08-01T15:23:26Z

jupyter_server/serverapp.py

+    def page_config_hook(self, handler, page_config):
+        page_config["allow_custom_env_variables"] = self.allow_custom_env_variables
+        return page_config
+


page_config is a concept specific to JupyterLab. Up until this point, Jupyter Server has been agnostic to it. This hook might need to land in jupyterlab_server instead, which extends jupyter_server and brings the page_config concept to the server.

Zsailer · 2024-08-01T15:26:42Z

jupyter_server/serverapp.py

+    allow_custom_env_variables = Bool(
+        False,
+        config=True,
+        help="""Allow to use custom env variables""",
+    )
+


I think this trait name and description is too vague.

Since this is specifically about passing custom environment variables to the kernel, I think we need to clarify in this trait and help message that we mean kernel env vars here.

Zsailer · 2024-08-01T15:33:22Z

Thanks for working on this @AnastasiaSliusar! Great stuff!

I glanced at the diff and left some minor comments in the PR. I know this is still in draft state, so I didn't do a deeper review.

Feel free to ping me here anytime you're ready for a review.

AnastasiaSliusar · 2024-08-01T16:08:22Z

Thanks for working on this @AnastasiaSliusar! Great stuff!

I glanced at the diff and left some minor comments in the PR. I know this is still in draft state, so I didn't do a deeper review.

Feel free to ping me here anytime you're ready for a review.

Hello, @Zsailer Thank you for your time and for your comments. I have a question. At the moment, custom env variables are not saved for a certain kernel. A user can just setup them for a kernel which has not be launched yet and then run a kernel. But if a user wants to run a notebook from a filebrowser a kernel won't run with its custom env variables if they have been setup before.
So my question is:

How do you think we should save custom env variables into notebook metadata or not?
I remember when we told about JEP parameterized kernel specs we made decision that saving user inputs into notebook metadata can violate the app security.

Thank you for your attention

Zsailer · 2024-08-01T16:52:55Z

I remember when we told about jupyter/enhancement-proposals#87 we made decision that saving user inputs into notebook metadata can violate the app security.

That's right. Notebook documents can leave the server they originated from, e.g. someone downloads their notebook and emails it to a friend. The environment variables (at best) include irrelevant information or (at worst) might contain sensitive information specific to the original server.

Aside from the security concerns, though, I just don't think these custom environment variables are a notebook-driven concept anyways, and thus, don't belong in the notebook document. They are essentially parameterized "instances" of the kernel. If we really want to cache previous launch configurations, we should do it through a separate service/extension IMO and store in a memory cache or (if persistence is needed across server lifetimes) store in a DB.

If you went that route, we'd need to extend JupyterLab's open document command, triggered by the filebrowser, to check this cache for previous launches and prompt the user to confirm if they'd like to use the same environment as previous launch.

AnastasiaSliusar · 2024-08-01T18:16:42Z

I remember when we told about jupyter/enhancement-proposals#87 we made decision that saving user inputs into notebook metadata can violate the app security.

That's right. Notebook documents can leave the server they originated from, e.g. someone downloads their notebook and emails it to a friend. The environment variables (at best) include irrelevant information or (at worst) might contain sensitive information specific to the original server.

Aside from the security concerns, though, I just don't think these custom environment variables are a notebook-driven concept anyways, and thus, don't belong in the notebook document. They are essentially parameterized "instances" of the kernel. If we really want to cache previous launch configurations, we should do it through a separate service/extension IMO and store in a memory cache or (if persistence is needed across server lifetimes) store in a DB.

If you went that route, we'd need to extend JupyterLab's open document command, triggered by the filebrowser, to check this cache for previous launches and prompt the user to confirm if they'd like to use the same environment as previous launch.

Thank you for detailed explanation

for more information, see https://pre-commit.ci

…iusar/jupyter_server into custom-env-variables

for more information, see https://pre-commit.ci

…iusar/jupyter_server into custom-env-variables

for more information, see https://pre-commit.ci

…iusar/jupyter_server into custom-env-variables

AnastasiaSliusar · 2024-08-09T11:28:28Z

Thanks for working on this @AnastasiaSliusar! Great stuff!

I glanced at the diff and left some minor comments in the PR. I know this is still in draft state, so I didn't do a deeper review.

Feel free to ping me here anytime you're ready for a review.

Dear @Zsailer I separated logic of using page_config between jupyter_server and jupyterlab_server (jupyterlab/jupyterlab_server#459) and you can review this PR now. Thank you for your attention:)

Zsailer

Thanks @AnastasiaSliusar.

I've left a few comments throughout the code. :) In general, I think we should use the new trait as a switch to accept/ignore these variables in the server REST API. Right now, this trait is only used to inform JupyterLab server clients about this feature via the page_config in your upstream jupyterlab_server PR. But that doesn't prevent clients from ignoring this config and still POSTing custom variables that affect the kernel. Ideally, if the trait is False, the server will ignore these variables even if a client tries to submit them.

One additional question. This PR adds the notion of custom kernel environment variables to the session REST API, but shouldn't we also extend this to the kernel's REST API too? Why special case it here? I think we should add this functionality to both APIs, since they are doing the same thing under the hood with kernels.

Zsailer · 2024-08-22T18:43:52Z

jupyter_server/services/sessions/handlers.py

@@ -77,6 +77,7 @@ async def post(self):
        kernel = model.get("kernel", {})
        kernel_name = kernel.get("name", None)
        kernel_id = kernel.get("id", None)
+        custom_env_vars = kernel.get("env", {})


Shouldn't this be conditional based on the value of allow_setup_custom_env_variables?

Dear @Zsailer I think it is a good idea. Sure, I agree. Thank you for suggestion.

Zsailer · 2024-08-22T18:45:33Z

jupyter_server/services/sessions/handlers.py

@@ -152,6 +154,8 @@ async def patch(self, session_id):
            changes["name"] = model["name"]
        if "type" in model:
            changes["type"] = model["type"]
+        if "env" in model:


Same question here, shouldn't this be conditional based on the value of allow_setup_custom_env_variables?

Zsailer · 2024-08-22T18:50:07Z

jupyter_server/serverapp.py

@@ -1427,6 +1427,12 @@ def _default_allow_remote(self) -> bool:
                        """,
    )

+    allow_setup_custom_env_variables = Bool(


A couple of comments on this new trait:

I think we should shorten the name if we can 😅

Even though I said we should shorten it, I think we need the word kernel in the name somewhere 😆 . How about accept_kernel_env_vars? It's slightly shorter and more clear (IMO).

I think this trait needs to be passed as a tornado setting so that the request handlers can use this trait too.

for more information, see https://pre-commit.ci

AnastasiaSliusar added 6 commits July 9, 2024 14:33

Add support of custom env variables

97747e6

Fix description of a flag

887f522

fix custom env vars

63e634e

Cleaned code

9355a31

Fix code styling

2ee8b19

Merge branch 'main' into custom-env-variables

86ddf23

Zsailer reviewed Aug 1, 2024

View reviewed changes

AnastasiaSliusar and others added 5 commits August 5, 2024 13:10

Fix type

ae37186

[pre-commit.ci] auto fixes from pre-commit.com hooks

93b14be

for more information, see https://pre-commit.ci

Fix types

4b04556

Resolve conflicts

ef07d15

Merge branch 'custom-env-variables' of https://github.com/AnastasiaSl…

084b5ba

…iusar/jupyter_server into custom-env-variables

SylvainCorlay added the enhancement label Aug 5, 2024

AnastasiaSliusar and others added 8 commits August 6, 2024 19:31

Remove custom page config

0c49734

[pre-commit.ci] auto fixes from pre-commit.com hooks

ba231eb

for more information, see https://pre-commit.ci

Add a test, clean code

56a6b42

Merge branch 'custom-env-variables' of https://github.com/AnastasiaSl…

b0feeba

…iusar/jupyter_server into custom-env-variables

[pre-commit.ci] auto fixes from pre-commit.com hooks

6bea16b

for more information, see https://pre-commit.ci

Fix formatting

90418ca

Merge branch 'custom-env-variables' of https://github.com/AnastasiaSl…

c1c84c8

…iusar/jupyter_server into custom-env-variables

Merge remote-tracking branch 'upstream/main' into custom-env-variables

9a9958a

AnastasiaSliusar marked this pull request as ready for review August 9, 2024 11:26

AnastasiaSliusar requested a review from Zsailer August 13, 2024 12:33

Update a variable name

f1c3d55

Fix lint

01db90a

Zsailer requested changes Aug 22, 2024

View reviewed changes

AnastasiaSliusar and others added 2 commits August 28, 2024 14:29

Fix setuping custom env vars

e51f647

[pre-commit.ci] auto fixes from pre-commit.com hooks

4d48ec4

for more information, see https://pre-commit.ci

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support of using custom env variables #1448

Add support of using custom env variables #1448

AnastasiaSliusar commented Aug 1, 2024 •

edited

Loading

Zsailer Aug 1, 2024

Zsailer Aug 1, 2024

Zsailer commented Aug 1, 2024

AnastasiaSliusar commented Aug 1, 2024

Zsailer commented Aug 1, 2024

AnastasiaSliusar commented Aug 1, 2024

AnastasiaSliusar commented Aug 9, 2024

Zsailer left a comment

Zsailer Aug 22, 2024

AnastasiaSliusar Aug 23, 2024

Zsailer Aug 22, 2024

Zsailer Aug 22, 2024

Add support of using custom env variables #1448

Are you sure you want to change the base?

Add support of using custom env variables #1448

Conversation

AnastasiaSliusar commented Aug 1, 2024 • edited Loading

References

Code changes

Zsailer Aug 1, 2024

Choose a reason for hiding this comment

Zsailer Aug 1, 2024

Choose a reason for hiding this comment

Zsailer commented Aug 1, 2024

AnastasiaSliusar commented Aug 1, 2024

Zsailer commented Aug 1, 2024

AnastasiaSliusar commented Aug 1, 2024

AnastasiaSliusar commented Aug 9, 2024

Zsailer left a comment

Choose a reason for hiding this comment

Zsailer Aug 22, 2024

Choose a reason for hiding this comment

AnastasiaSliusar Aug 23, 2024

Choose a reason for hiding this comment

Zsailer Aug 22, 2024

Choose a reason for hiding this comment

Zsailer Aug 22, 2024

Choose a reason for hiding this comment

AnastasiaSliusar commented Aug 1, 2024 •

edited

Loading