Support notebook 4.3 token generation (or equivalent)

[parente]

We can expose the 4.3 token feature. Or it might be simpler to implement the same random token generation scheme and use the auth handlers already mixed into the various classes here.

The main significant difference between what's here and in nb 4.3 is the support for tokens as query params in addition to headers. This allows the use of a token on the websocket URL. It should be easy to support that here as well.

I've one worry: the further diverging code bases between KG and notebook.

[parente]

The tokens in notebook as for one-time use, with the traditional cookie serving as the proof of authentication thereafter. That won't work for KG use cases where a cookie jar may not be available.

I think having an option to generate a secure token if one is not provided, and accepting that token as a URL parameter, especially on web socket connections, is the way to go.

[rolweber]

I've just been trying to connect to a kernel gateway with auth token through a node.js program using the jupyterlab/services client. I can list kernel specs and running kernels. But when the web sockets come into play, after starting a kernel or when trying to execute code, things fail. The client reports loss of connection, and the kernel gateway a write error. I assume that this is because the kernel gateway doesn't understand the token being passed as a URL parameter to the web socket?

client output:

Starting websocket ws://localhost:8888/api/kernels/9e5be78e-91eb-498f-865a-e5fde5ce192f
connected
Kernel: reconnecting (9e5be78e-91eb-498f-865a-e5fde5ce192f)
Connection lost, reconnecting in 1 seconds.
Starting websocket ws://localhost:8888/api/kernels/9e5be78e-91eb-498f-865a-e5fde5ce192f
Connection lost, reconnecting in 2 seconds.

kernel gateway output:

[W 170303 09:21:07 web:1971] 401 GET /api/kernels/9e5be78e-91eb-498f-865a-e5fde5ce192f/channels?session_id=ce11007d55f7c30eff0d5caee8f97cca&token=Oh8Fifteen (172.17.0.1) 3.75ms
[KernelGatewayApp] Initializing websocket connection /api/kernels/9e5be78e-91eb-498f-865a-e5fde5ce192f/channels
[KernelGatewayApp] Using contents: services/contents
[E 170303 09:21:08 web:996] Uncaught exception in write_error
    Traceback (most recent call last):
      File "/opt/conda/envs/jupyter/lib/python3.5/site-packages/tornado/web.py", line 994, in send_error
        self.write_error(status_code, **kwargs)
      File "/opt/conda/envs/jupyter/lib/python3.5/site-packages/notebook/base/handlers.py", line 410, in write_error
        html = self.render_template('%s.html' % status_code, **ns)
      File "/opt/conda/envs/jupyter/lib/python3.5/site-packages/notebook/base/handlers.py", line 343, in render_template
        template = self.get_template(name)
      File "/opt/conda/envs/jupyter/lib/python3.5/site-packages/notebook/base/handlers.py", line 339, in get_template
        return self.settings['jinja2_env'].get_template(name)
    KeyError: 'jinja2_env'

Would it be possible to fix just the authentication part, without introducing token generation, in the 1.x series? I don't know to what degree the breaking API changes in 2.0 would affect our planned setup.
I'd appreciate a pointer to the relevant sections of the code. I've got the approval to contribute to Jupyter as an IBM employee now, so I could work on a PR :-)

Another direction I was pondering is to enable basic auth alongside token auth in KG. The web socket authentication with basic auth works against Bluemix Spark as a Service when username and password are encoded in the URL. By enabling basic auth in KG proper, we could use the same trick without an authentication proxy.

[rolweber]

This seems to be the place...
https://github.com/jupyter/kernel_gateway/blob/317033f90e17af0a9d314b78fba7941e968a5eeb/kernel_gateway/mixins.py#L57

And the corresponding code in notebook...
https://github.com/jupyter/notebook/blob/68a514a29d0e79af66ce6dd75c87e991fb58126d/notebook/auth/login.py#L112

So there is no distinction between web socket and plain request authentication. Both will work with a token either in the Authorization header or as a URL parameter. A token passed as URL parameter takes precedence over one passed in the Authorization header. From a gut feeling, I would have done it the other way round.

[parente]

The current KG release doesn't understand tokens for websockets yet.

[parente]

I think the only thing required to close this out is the option of automatically generating a token. We could take a security-first approach in 2.1 and start doing that by default if the user does not specify a blank or explicit token. This is the approach that jupyter/notebook took in 4.3. Or, we can remain backward compatible with existing kernel gateway apps and insecure by default, yet have a way for a user to specify that a random token should be generated at startup.

I think security-first plus keeping the KG and original notebook server as closely aligned as possible is the way to go.