Come up with clear policy around how integrations are managed

[adamkpickering]

As it is, there is confusion as to how we want to manage integrations. By "integrations", I mean "the programs we install alongside Rancher Desktop to enable users to use the services running in the VM." So, for example, kubectl and helm.

There are a number of issues related to this:

#848
#889
#1300
#1400

I am creating this discussion so that we can establish a clear policy on how we manage integrations.

[adamkpickering]

First I want to talk about Jan's idea. His idea is to have a directory that contains the integrations. I'm not sure whether this directory is one that exists as part of the installation of RD, or whether this directory would be created at some point when RD is running. We would then create an entry in /etc/profile.d/ that adds this directory to the users' PATH.

While I like this scheme because of its simplicity, I don't see a way for it to work for AppImages on Linux. This is because of how AppImages work: they are a filesystem image that is mounted using FUSE at runtime on a directory in /tmp/. This directory's name differs every time the AppImage runs. So, there is no way to put a directory containing the integrations on the PATH in the case of an AppImage. Of course, one could create an external directory and make symlinks in it pointing to the integrations in the mounted AppImage directory, but at that point we might as well continue to use symlinks from the mounted AppImage directory to a directory in the PATH.

Another possible problem is users overwriting PATH in various ways. I'm ashamed to admit that I currently do this in my .bashrc. As incorrect as it may be, it is an example of what could go wrong with modifying the PATH variable.

Of course, there is no reason that we couldn't have one scheme for AppImages and another scheme for other formats. But it would be nice to minimize the number of schemes we have.

[jandubois]

First I want to talk about Jan's idea. His idea is to have a directory that contains the integrations. I'm not sure whether this directory is one that exists as part of the installation of RD, or whether this directory would be created at some point when RD is running. We would then create an entry in /etc/profile.d/ that adds this directory to the users' PATH.

Yes, it is a directory that exists inside the app already:

$ ls /Applications/Rancher\ Desktop.app/Contents/Resources/resources/darwin/bin
docker  helm    kubectl kuberlr nerdctl

All you have to do to put it on the PATH (for default shell configurations) is:

$ echo "/Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/bin" | sudo tee /etc/paths.d/rancher-desktop
/Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/bin

Various shells will then include it in PATH via

$ grep -C 1 path_helper /etc/profile

if [ -x /usr/libexec/path_helper ]; then
	eval `/usr/libexec/path_helper -s`
fi

Note that we already do exactly the same thing on Windows; there are no symlinks to configure, but these two directories are appended to the end of the PATH:

C:\Users\Jan\AppData\Local\Programs\Rancher Desktop\resources\resources\win32\bin
C:\Users\Jan\AppData\Local\Programs\Rancher Desktop\resources\resources\linux\bin

We append the Linux versions as well, so they are automatically on the PATH inside the WSL2 distros, which ignore the Windows versions unless you invoke them with an explicit .exe extension, e.g. kubectl.exe get no will invoke the Windows version even from inside WSL2.

We should have documentation for this, so people who overwrite their PATH manually (like Adam and me) can implement this themselves. Or people using "weird" shells that may not invoke path_helper by default.

In addition we could have some warnings when we can detect that our utilities are being shadowed by other directories earlier on the PATH. That may be a can of worms though, and should be a separate issue for discussion.

[adamkpickering]

I do like this approach for macOS, other than the need for a sudo. Let's call it approach 1. It seems that the only alternatives on macOS are:
2. continue doing it the current way (managing symlinks in /usr/local/bin/, which doesn't require sudo if homebrew is installed if Matt's comment is correct).
3. creating symlinks, but with the symlinks in ~/bin/ rather than /usr/local/bin/; again, this requires modification of PATH since ~/bin/ isn't in it by default.

Which of the three is the best depends on:

• How important is it that we keep all of the data local to the user?
• How important is it that we avoid sudo calls?

Of course, all of the above doesn't apply to Linux. If we do anything other than what we are currently doing (approach 2), I'd say there will be enough subtle differences to warrant separate implementations of integration management for macOS and Linux.

[jsoref]

Personally, I wouldn't mind the macOS sudo creating a symlink from /etc/paths.d/rancher to a ~/paths.d/rancher file.

If it exists, then Rancher wouldn't need sudo going forward.

Yes, it isn't "secure", but as an option, I'd be happier w/ that than constantly being asked for sudo credentials.

[jandubois]

I just remembered that I suggested using /etc/paths.d before (in one of the very first issues on this repo): #10 (comment). I'm not giving up easily! 😸

And the warning about shadowing binaries on PATH is already implemented for our current system: #228

[adamkpickering]

I want to describe the problem that initially got me onto this subject - maybe it will help others understand where I'm coming from.

The problem has to do with the AppImage on Linux. I'm going to describe how AppImages work again because I want to have all the information here. An AppImage is an executable with a filesystem image embedded in it. This filesystem image contains everything that is needed to run an application: the application executable, .so's, static files, and so on. The executable part of the AppImage has just enough code to:

1. Mount the filesystem image on a location like /tmp/.mount_rancheU3PI1I/.
2. Exec into the application executable, which is inside the filesystem that is now mounted.

This is great because it means that we can ship and run our programs as a single file. All the user has to do is make the AppImage executable, and they have a program that just runs on their machine. Of course, in our case it gets a little more complicated.

Running the application from a mounted filesystem image means that when the application exits, the filesystem image must be unmounted. This is important to understand because we rely on the filesystem image to provide our integrations: symlinks are made in ~/.local/bin/ and point to the integrations in /tmp/<mount_directory>/opt/rancher-desktop/resources/resources/linux/bin/. So assuming we've made a bunch of symlinks to this mount directory while running, once it exits, we will be left with a bunch of dangling symlinks.

This is a problem that David noticed previously. The solution at that time (see background.ts:281) was to delete these symlinks when the user closes RD (I've also noticed that docker is not removed here, since this code predates the docker integration). This is an imperfect solution though: once the user starts RD again, the needed symlinks will not be present. This is because the symlinks are only created on the first run (see background.ts:141). The user is left with the bad experience of having to check the boxes in the Supporting Utilities tab on every startup to get working integrations.

Here is what I propose. I want to create a part of the Settings object that is for tracking which integrations we are managing, and which we are not. These would all be true by default, but set to false under certain conditions (see the flowchart). I want to run a check on every startup and every shutdown - I've recorded the logic in this flowchart.

I'm not yet sure how this will work with the existing PathConflictManager. We may be able to do away with it, or maybe integrate it in some way. I'm also not sure what this would look like for Windows (seems fine already?) and macOS (maybe we will do something different?). But it's Friday and my brain is tired - I'll come back to this on Monday.

[adamkpickering]

Regarding PathConflictManager, I have an idea of what to do. Right now, if a path conflict is detected by PathConflictManager, it sends a k8s-integration-warnings event to the renderer process. The renderer process then grays out the checkbox for the integration that has the conflict, so that the user cannot check or uncheck it. I propose a slightly different way of doing it: if a box is checked (which corresponds to a true state of the integration in settings.integrations), then create a pop up that allows the user to choose either:

1. Delete the file that caused the conflict and continue to have RD manage that integration
2. Keep the file and set the state of the integration in settings.integration to false

If the box is not checked, we don't have to worry about it, since that means that RD is not responsible for managing that integration.

[jsoref]

I'd imagine that instead of creating symlinks one could create a single shell script (possibly bash, but possibly posix) and a bunch of symlinks to that shell script.

The script would act as a shim, and it would check to see if Rancher is running, if it is, it could find the current path to the mount, and then run the binary it's supposed to be (based on $0) and pass its arguments $@.

~/.rd/bin/rancher-desktop-appimage-shim

~/.local/bin/docker -> ~/.rd/bin/rancher-desktop-appimage-shim
~/.local/bin/nerdctl -> ~/.rd/bin/rancher-desktop-appimage-shim
...

How the script chooses to behave if rancher desktop isn't running is open for discussion -- it could try to launch rancher, wait for it to arrive, and then do the normal work, or it could print an error (to stderr) suggesting that the user start rancher desktop and then exit w/ an exit code.

[jandubois]

I feel like managing each integration individually is just too much, so I would prefer to have the same approach for AppImage as for macOS and Windows (and non-AppImage Linux):

Whenever Rancher Desktop starts, it can will delete (if it exists) and create a symlink from /tmp/.mount_ranche*/.... Then the user can put that directory anywhere in the PATH they want to.

Sample commands (but on macOS):

$ rm ~/.rancher-desktop

$ ln -s "/Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/bin" ~/.rancher-desktop

$ ls ~/.rancher-desktop/
docker  err     helm    kubectl kuberlr nerdctl

$ ~/.rancher-desktop/kuberlr version
kuberlr version: 0.4.1 (tagged as "v0.4.1") 2021-08-17T16:44:50Z go1.13.15

Just like on other platforms, users can then add their own version of utilities on the front before that, to shadow the versions included with Rancher Desktop.

The AppImage should delete the symlink when shutting down, but I don't think a dangling symlink in PATH is a problem (needs testing).

[jsoref]

Having dangling directories in the path is a bad idea. Empty directories are fine, dangling ones will result in something screaming eventually. (I can't remember which things explode when this happens, but I'm fairly confident I've seen things explode over this.) And, it's an invitation for a user (or program) to accidentally create a file in that location where you're expecting to have a directory and then all sorts of problems occur -- can you safely delete the file?

[jandubois]

One complication is that the docker plugins (buildx and compose) are not being installed on the PATH but into ~/.docker/cli-plugins.

It is not clear to me yet how to handle this.

@ericpromislow Is there any problem with installing the plugins into the PATH as docker-buildx and docker-compose?

[adamkpickering]

So Jan and I chatted about this and we'd tentatively like to move in the direction Jan described, that is, modifying PATH to point to the directory (that is a part of the RD install) that the integrations are in. The exception is AppImages, where upon startup RD would create a directory that is a symlink to the AppImage's current integrations directory (since this directory changes location every startup) and remove that symlink on shutdown. In this case PATH would be modified to contain the path where we create the symlinked directory.

There are a couple wrinkles though:

1. We need to be able to install docker CLI plugins. Unfortunately, it may be hardcoded to look in ~/.docker/cli-plugins/ for CLI plugins. I have a work item in Stripey for doing some research about what is possible here.

2. We would need to modify the user's PATH somehow. It sounds like the mechanism to do this on macOS is modification of /etc/paths.d/. The mechanism is uncertain on Linux - it may be possible to do what we need through /etc/profile.d/ or ~/.profile. /etc/environment is there, but we don't want to modify a file, especially one so important to the functioning of the system.

3. At least on macOS, sudo access is required to make modifications to /etc/paths/d. Jan suggested that we can equate allowing us to have sudo privileges to an "easy button": if the user wants to trust RD to set things up, they can give RD sudo access and it can configure things for them. If not, they can make their own modifications to PATH, or install needed programs themselves.

4. The user may potentially already have, or while using RD install, a program at a location in PATH that causes it to shadow our version. The user would end up using this other version rather than the one associated with RD. We need some way of notifying the user if this is the case (see below).

5. This would require large changes to the UI. Specifically, it means the removal of the Supporting Utilities tab (which already doesn't exist on Windows). And if the Supporting Utilities tab is gone, we need a place to notify the user if our integrations are being shadowed by some other executable in their PATH. Jan suggested that this be placed in the Troubleshooting tab. Ideally we want a situation where the user can be notified that this is a potential issue, but they can turn off any annoying notifications/warnings if this is a situation they want to have.

[adamkpickering]

@mook-as @mattfarina what do you think?

[mook-as]

Unordered thoughts:

• We'll need to significantly change the UI in this area anyway.
• On systemd-using Linux, there's ~/.config/environment.d/ that we can use. I'm not certain we'd be able to place our tools in any sane order in there, though.
• On mac I don't know of similar things that are per-user (man path_helper didn't turn up anything useful).

[mattfarina]

We originally created the setup we have today because we were trying to avoid a requirement on sudo. Things have changed since then.

We need to be sure we tell the user what we are doing and why we are doing it.

I wonder how many users don't want to use admin credentials or don't have that level of access. That's really a question for another time.

[jsoref]

From memory, from another life (when @mook-as and I worked in the same realm), I created my own path_helper that emulated the behavior of the system path_helper but worked on items in ~ and then I stuck that into my profile or similar.

[adamkpickering]

I looked into whether the directory docker looks in for CLI plugins is configurable. It is, kind of. The way to change it is through the DOCKER_CONFIG environment variable, which changes the directory in which all config (not just CLI plugins) is stored. By default this directory is ~/.docker/. So we can change it, we just have to worry about how other things are affected.

Maybe we can just symlink a docker plugins directory within the RD installation to ~/.docker/cli-plugins/...

[jandubois]

Question: should we always create a symlink in ~/.rancher-desktop/bin, and not just for AppImage?

Pro:

• It keeps the PATH shorter.
• It works the same across different install options, so simplifies the docs. Less things users must understand.

I would say we should definitely do this for all versions on Linux. I'm less sure about macOS.

On Windows we have 2 directories, making it more complex, and we already have a way to put them on the PATH, so it seems not helpful there...

Bikeshedding: I changed the symlink location to ~/.rancher-desktop/bin so we can also move the LIMA_HOME to ~/.rancher-desktop/lima (see #1465). For this purpose it would be preferable to keep it as short as possible, so ~/.rd/lima would be better (but we use rancher-desktop everywhere else).

[adamkpickering]

I agree that we should always create a symlink in ~/.rancher-desktop/bin on Linux. As you say, it simplifies everything even if it isn't necessary for RD installed via deb or rpm.

As for macOS, I don't think it matters much either way. But we could reuse some code, and maybe some documentation, if we make it work the same as Linux.

Maybe we can just leave Windows the way it is? It seems to be fine the way it is, though I haven't tried it.

I think we should not use the name ~/.rd/, because:

1. It is surprising: as you say, we use rancher-desktop everywhere else.
2. It is short enough that a collision with some other program is possible, though to be fair I don't know of any such program.
3. It isn't as easily identifiable as having something to do with RD.

[jandubois]

Personally I'm not a fan of this, but for the sake of discussion:

Should we offer to edit shell configuration scripts? Unlike editing /etc/* files, this does not require sudo:

Look through the user's home directory for .bashrc, .bash_profile, .profile, .zshrc etc and offer to add our directory to the PATH in there. It would just append or replace a block:

### RANCHER DESKTOP START (DO NOT EDIT)
export PATH="$HOME/.rd/bin:$PATH"
### RANCHER DESKTOP END (DO NOT EDIT)

Bonus: we likely end up close to the front of the PATH for inexperienced users, so are less likely to be shadowed. Experienced users will not use this at all, or then edit the settings to their liking.

[DanHam]

I do not like anything writing to my carefully curated and lovingly crafted dotfiles.

Too many applications nowadays feel they have the right to step in and smash their configuration over the top of everything else (some do this with out even telling the user what they are doing) all in the name of making things easy for the user.

I would argue that this actually makes things harder for the user as they have no control over what is automatically written into those files.

In addition, many users store their dotfiles in version control. There is the danger that a snippet added to a dotfile could be written into version control and later restored when the need to have the snippet was no longer there.

Please don't do this.

[DanHam]

My 2 cents as an end user...

Clearly, there has been a lot of discussion here about the possible ways to go forward with this.

So that you know where I am coming from - I am on Fedora 35 and use the AppImage to run RD (RD is great by the way!).

I opened #1619 as I wanted the preferences I had selected for 'Supporting Utilities' to be preserved across restarts of RD. This was already mentioned in the comment above:

The user is left with the bad experience of having to check the boxes in the Supporting Utilities tab on every startup to get working integrations.

Other than that issue (and the issues/bugs picked up at the head of this discussion) the current way things work is intuitive and work well for me. I have a clear understanding of what selecting those check boxes does and how it works - the implementation is simple enough for me to understand.
Perhaps there should be a note on that screen telling users that 'For this to work .local/bin must be included at the head of your path' - but that's about all the improvement I think this needs.

Some of the implementation details/internals you are discussing are clearly beyond what I need to know about or understand as an end user. However, I would like to say that I do not like any solution that:

• Messes with my path
• Messes with my dotfiles
• Requires sudo
• Is too complicated for me to understand 🙂

I hope that helps.