using rootlesskit with preconfigured network namespace

[chri2]

I'd like to achieve the following setup for users who start docker rootless: docker uses a network config that is statically configured by root and cannot be changed by the unprivileged user who started rootlesskit (and docker or whatever inside it).

Motivation: building a network between dockerd running for different users that cannot be changed by the users running the docker daemons to

• avoid an additional layer of software (like slirp4netns)
• separate different services run inside docker using different users for them (e.g. a 'frontend' user to run some proxy services for incoming connections, a 'proxy' user to validate the requests and forward them to backends, a 'backend' user to run backends containing data to be protected)
• let only root change the network setup provided to the different users and define routing and firewalling between them

So far I came up with the following approaches:

1. preconfigure named network namespaces during system setup and have rootlesskit enter the appropriate namespace configured for the user it is running in
   I didn't find a way to setup a network namespace as root and provide access to an unprivileged user.
   If this would be possible it would be my preferred solution and I'd kindly ask to extend rootlesskit with an option to enter an existing network namespace instead of using the hosts network or vpnkit or slirp4netns.
2. let rootlesskit make a new unconfigured namespace that is configured by the system when it comes into existence (signalling over dbus or - if there's a mechanism - between user-systemd and systems-systemd)
   Do not like this idea much, because timing and dependencies in the order of thing happening from different services are involved.
   A variant could be to let users systemd start docker in a network namespace by configuring the service with "PrivateNetwork=".
3. put every user inside its own network namespace by configuring the system
   Maybe via configuring the users systemd and logind to run inside a network namespace.
   This is a variant of 2. and thereby the same disadvantage applies in my opinion. Furthermore maintaining the containers run by docker becomes more difficult (maybe secure, also?), because the user running the docker is restricted to the same network restrictions as docker and its containers.
4. while writing this: security-wise a combination of 1. (preconfigured named network namespaces per user) and 3. (restricting each user to its assigned network namespace) might be the most secure scenario.

I'd appreciate opinions for this idea, the approaches I found and help on how to configure whatever makes sense.

[chri2]

I followed my idea number 4 and preconfigured a netns as root and setup pam to put all processes of a test user into that netns. Trying to start dockerd in a rootlesskit environment with host network (which for that user is a preconfigured netns) I ran into the following problen which is reproducible for any user in the main/default netns, also:

I called dockerd-rootless.sh like this:

    /usr/local/bin/dockerd-rootless.sh --debug --iptables=false --bridge=none --ip-forward=false \
    --ip-masq=false

When I start a busybox docker container I get the error message "docker: Error response from daemon: Could not check if docker-default AppArmor profile was loaded: open /sys/kernel/security/apparmor/profiles: permission denied."

I found (using strace) that dockerd does not try to look for /sys/kernel/security/apparmor/profiles when rootlesskit is called without the options to use host networking.

1. Why does dockerd look for apparmor when rootlesskit is started without creating a netns?
2. How does rootlesskit alter the environment so that dockerd does not look for apparmor?
   Found https://github.com/moby/moby/blob/68bec0fcf7a5eeb59c027287d06598098edc9f2c/vendor/github.com/containerd/containerd/pkg/apparmor/apparmor.go#L37, which seems to be the place where dockerd checks whether to look for apparmor. Setting container=not_empty as an environment variable before dockerd is called is a workaround, but not the answer to the question.

[AkihiroSuda]

RootlessKit mounts a new /sys when the network namespace is unshared, so dockerd avoids looking up apparmor stuff

[chri2]

Thanks for the answer!

So, rootlesskit does not mount a new /sys when the network namespace is not setup, because "--net host" is used?

If so and if done intentionally, why? If not so, why does dockerd in "rootlesskit --net host" complain about AppAmor?

[AkihiroSuda]

If so and if done intentionally, why?

Not my own choice. The kernel is designed to require unsharing the network namespace for mounting a new /sys fs.