Add support for Linux network namespaces to the socket abstraction

[tonya11en]

This proposes extending SocketAddress to allow us to specify the network namespace. The primary use-case I have in mind is one where we'd have listeners that listen in different network namespaces than the one that the worker threads belong to.

This would not change any existing behavior if the field is unset.

The API changes I have in mind can be found in this draft PR:
#38945

This should not require a change to the threading model, as #29675 suggests. The way we do this should be something like:

1. Spawn a new thread and change its namespace via setns().
2. Pass the socket fd back to the primary thread.
3. You now have a socket in another netns that you can accept connections on.

For sockets that specify another netns, this makes initial socket creation a syscall-heavy operation, but for the use-case above I believe it should only occur during listener instantiation.

Relevant Links:

• https://man7.org/linux/man-pages/man7/network_namespaces.7.html

[tonya11en]

attn: @envoyproxy/api-shepherds

[abeyad]

To make sure I understand, are you saying if network_namespace_filepath is set: (1) spawn a new thread (2) call setns() to change the namespace (3) create a socket in this thread (4) pass the socket FD to the worker thread that spawned this new thread (5) terminate the newly spawned thread (6) use the socket FD in the specific namespace from the existing worker thread?

To start with, a couple questions:
(1) What are the benefits / use cases for wanting to do this?
(2) Are there any caveats or behavioral differences in using a file descriptor belonging to a namespace from a thread belonging to a different namespace? Any security or permissions issues that could arise?

[tonya11en]

@abeyad your understanding is correct.

The use case I have is one where we'd have listeners that listen in different network namespaces than the one that the worker threads belong to. There are others, such as communicating with endpoints in different net namespaces or opening connections originating from specific net namespaces.

Using a file descriptor for a socket belonging to a different network namespace is fine, since the network namespace doesn't control anything related to file descriptors. There are other kinds of namespaces that this may not be true for, but that's irrelevant to the change I'm proposing.

Regarding the question about security/permissions issues, I'm not sure what to say other than we're not undermining any of the existing security guarantees. Doing this requires NET_ADMIN capabilities for all of the network namespaces involved, otherwise you run into "permission denied" errors. If users have security concerns related to something like this, then they would simply not give Envoy NET_ADMIN capabilities.

[abeyad]

Thanks for your reply @tonya11en !

The use case I have is one where we'd have listeners that listen in different network namespaces than the one that the worker threads belong to.

I was mainly curious what the use cases for this actually are - when would you want to have listeners listen in different network namespaces than the worker thread's namespace? If Envoy is running in a container, I assume the worker threads and sockets all get the same namespace. I just wasn't understanding what the situations or use cases are for when the network namespace might be different from the worker thread's namespace (probably due to my own lack of experience dealing with namespaces). I assume you don't want to just call unshare() in the worker thread as that would change the worker thread's associated namespace.

[tonya11en]

@abeyad I need Envoy to listen on ports in other containers' namespaces. In my situation, I have a single Envoy for my k8s node. A pod's egress mesh traffic cannot enter the host-side veth (other end of eth0) because Cilium intercepts the traffic at the "TC ingress" hookpoint. That hookpoint catches all packets as they enter the kernel's network stack, so I can't redirect the traffic via TPROXY rules. I can't do anything about this Cilium hook point for various reasons.

I don't want to attach an XDP program to the interface. I cannot bridge the pod and Envoy net namespaces with a veth pair and routing rules because I'd need to assign an IP address to the interfaces. I also don't want to create a bridge interface for performance reasons.

[abeyad]

Thanks for the explanation @tonya11en , now I see what you were trying to achieve. In that case, the proposal and API LGTM.

[codablock]

I have another use case for this: In my project, I'm using libcontainer to spawn VM-like sandboxes. The process that spawns these containers, needs to direct selected host-traffic into the sandboxes. My current way to achieve this is via DNAT which sends the the traffic into the sandbox via a veth-pair, where envoy is listening inside the network namespace. This however requires custom iptable rules, which are not fun to automate and maintain.

If envoy would be able to listen on the host network namespace and run its worker thread in the sandbox namespace, I'd be able to use plain envoy proxy.

I have already implemented the same pattern for simple a simple custom-built TCP and UDP proxy, but this obviously lacks quite a lot of necessary features.

[tonya11en]

Nice, it's good to see other use cases. I've already written the code for this and am using it for some POCs at the moment. The PR should be sent out with docs/tests in the next week or so.