[Spike] Determine what needs to happen for binderhub-service to expose the binderhub UI

[yuvipanda]

Currently, the binderhub-service chart primarily exposes the binderhub API. Given that we're not going to be using the upstream binderhub chart (#4085), we should explore what is needed for binderhub-service to expose the binderhub UI.

This should be done keeping in mind that we don't want to tie assumptions about how jupyterhub is deployed into this binderhub. The outcome should be primarily documentation, as well as general extensibility improvements to binderhub-service.

Tasks

• Schedule - Figure out who needs to be apart of this and the duration for the spike
• Do the spike
• Report the outcome

---

Spike outcome

1. Actual spike

@consideRatio, @yuvipanda and @GeorgianaElena met for the spike and after the meeting the following steps were considered that need to happen for a binderhub ui style hub with binderhub-service (this is summarized from https://hackmd.io/G06D9YfOSsSf_mt13EeZoA?both):

• the jupyterhub is dedicated to be ephemeral (no persistent home dirs).
• binderhub server software needs to be aware about:
  • domain names
    • needs both local and public in order to redirect the user from web frontend
  • have an jupyterhub api token
    • autogenerated and exposed in a Secret that we can mount assuming its in the same namespace
    • Z2JH should define a service for binderhub
    • this needs additional work in binderhub-service
• BinderSpawnerMixin needs to be in the hub image

2. After-spike conclusion

Have a binderhub architecture walk-though for @GeorgianaElena and other interested 2i2c team members to spread the knowledge

• Run a knowledge sharing session about binderhub technical architecture #4113

3. After-spike work session

@consideRatio and @GeorgianaElena met after the spike for a working session.
@GeorgianaElena's rough plan before the working session was to follow the instructions @yuvipanda wrote for installing and connecting binderhub-service https://binderhub-service.readthedocs.io/en/latest/tutorials/install.html to a hub and from there, go with the plan @yuvipanda and Erik put toghether during the spike (referenced at point 1).

But:

• Georgiana's local install of minikube was erroring out and needed fidelling
• we had deleted the previous gcp project that we used to play around with binderhub-service, so that couldn't be used either
• Erik was concerned about needing to deal with the complexities of creating a registry/repo to push the built images to and was reasoning about the difference between a registry and a repository that were not clear in the docs

So we concluded to:

• try and prototype something under the 2i2c infrastructure
• reason about what are the configuration changes that we are doing and are specific to 2i2c infrastructure only
• try to document everything for a generic hub deployment once we have the 2i2c setup figured out

We:

• started by copy-pasting configuration from the binderhub chart, reasoning about what we need and what we don't
• also did a check on what we had in the hackmd originally and what we ended up doing during the session and the hackmd plan was pretty accurate, but definitely not captured everything.
• identified some limitations in the binderhub-service chart and Erik ended up opening a bunch of PRs, adding an extraEnv option to the binderhub-service deployment so we can set things like the hub api token.

We are not done yet, but prototyping this new hub helped identify some missing steps.

Outcome of that session is #4119

[yuvipanda]

I'd like this to involve all the folks who have worked on this so far, which is me @consideRatio and @GeorgianaElena. Let's at least do a 1hr session with us three, and then we decide on more split sync sessions as needed. I propose 7AM pacific on Tuesday, May 21. Does that work for you both, @consideRatio and @GeorgianaElena?

[GeorgianaElena]

@yuvipanda, yes, that works for me. Thank you

[consideRatio]

@yuvipanda yes that works well!

[yuvipanda]

Great, I've sent an invite.

[yuvipanda]

Notes from today's meeting: https://hackmd.io/G06D9YfOSsSf_mt13EeZoA?both

Next step is for @consideRatio and @GeorgianaElena to have a meeting between themselves to validate these requirements by testing it out locally, trying to set up a connection between binderhub-staging and a jupyterhub. @GeorgianaElena is to drive this, to make sure that @consideRatio doesn't become an engineering bottleneck.

[yuvipanda]

The outcome of that meeting would be to confirm the work needed to connect a binderhub-service instance to a jupyterhub instance next to it, on the same namespace. We have a draft of what we think is needed in the hackmd.

[consideRatio]

This is code of relevance from the binderhub chart, in how it re-configures the jupyterhub chart: https://github.com/jupyterhub/binderhub/blob/c6c5dc8fe73f81ca538c47b420b33f317c3aa8ae/helm-chart/binderhub/values.yaml#L65-L255

1. JupyterHub.spawner_class
   Needs to be adjusted to reference a class including the BinderSpawnerMixin.
2. cmd=...
3. cull
   Should jupyterhub users be deleted over time? This isn't of relevance for binderhub-service in either mode I think, but could be of relevance to the truly public hub with nullauthenticator.

To enable binderhub-ui we need to also override binderhub-service helm chart defaults:

1. config.BinderHub.hub_url=false (was it named like this?)
2. config.BinderHub.binder_badge_url=false (was it named like this?)
3. config.BinderHub.enable_api_only_mode=false
4. config.BinderHub.auth_enabled=true
   Also https://binderhub.readthedocs.io/en/latest/authentication.html stuff.

[consideRatio]

Me and @GeorgianaElena met and worked for more than five hours to see how far we'd get in deploying something.

• [New Hub prototype] Binderhub UI demo #4119
• Several PRs in binderhub-service

[GeorgianaElena]

I've updated the top comment with a more detailed report of the spike and of the discussions and working session that happened afterwards.

I think with that we can close the spike and track the WIP PR in the new hub issue that Yuvi created #4133. Thank you!