[NixOS] Handling multi-layer and parent symlinks for /etc/resolv.conf

[KiruyaMomochi]

The pull request #318 adds support for escaping resolv.conf symlinks.
However, I've encountered some issues in more complex environments like NixOS, where symlinks can have multiple layers and even parent folders can be symlinked.

Multi-layer symbolic link

NixOS often generates files under /etc from a config file and places them in /nix/store, symlinking them back to /etc.

This can result in multi-layer symlinks like:

/etc/resolv.conf
  -> /nix/store/random-hash/etc/resolv.conf
  -> /run/systemd/resolve/stub-resolv.conf

In my humble opinion, one of the solution could be mounting the realpath to the first unresolvable layer of symlink.

Parent folder is also symbolic link

The real case is not that simple.
The file /etc/resolv.conf is actually a symlink to /etc/static/resolv.conf.
It is /etc/static that being symlinked to /nix/store/:

/etc/static
  -> /nix/store/random-hash/etc/

/etc/resolv.conf
  -> /etc/static/resolv.conf
 (-> /nix/store/random-hash/etc/resolv.conf)
  -> /run/systemd/resolve/stub-resolv.conf

This makes it difficult to resolve the actual path using readlink or realpath.

Possible Solutions and Workarounds

1. Check readability after pivot_root:
   At the very least, I think we may add a check after pivot_root to warn users if /etc/resolv.conf isn't readable.
   This could provide insights into DNS resolution issues.
2. Bind mounting all layers and parent paths:
   We may need to find paths to do bind mount from every layers of the symlink, and also each parent paths.
   This could potentially ensure resolvability after switching root.
3. Use a different syscall that can mount over a symlink.
   It will be easier if we can bind mount to /etc/resolv.conf directly. During searching I have found MS_NOSYMFOLLOW in mount(2), but that does not work for me. I have also found https://serverfault.com/a/1126837/979197, which suggests using AT_SYMLINK_NOFOLLOW.

As a alternative, it's also possible to patch slirp4ns in Nix package registry to additionly bind-mount /nix into the sandbox.

[AkihiroSuda]

Possible Solutions and Workarounds

4. Just copy the resolv.conf file, instead of mounting it

cc @giuseppe

[AkihiroSuda]

@KiruyaMomochi Do you plan to submit a PR?

[ktoulliou]

Same problem with Ubuntu 22.04.5 LTS with a full NetworkManager+systemd-resolve configuration.

As I'm not root, I can't modify this symbolic link /etc/resolv.conf -> /run/systemd/resolve/resolv.conf (as explained in many open/closed tickets).

In my docker rootless container, I have a 10.0.2.3 name server in /etc/resolv.conf (internet works with IP but not with DNS).
Also my docker pull doesn't work (error dns).

Do you have a workaround without using root rights?