Support per-path unix socket allowlisting on Linux - #510
Open
shawnm-anthropic wants to merge 3 commits into
Open
Support per-path unix socket allowlisting on Linux#510shawnm-anthropic wants to merge 3 commits into
shawnm-anthropic wants to merge 3 commits into
Conversation
network.allowUnixSockets was macOS-only. On Linux the seccomp filter denies
socket(AF_UNIX, ...) outright, so a program that talks to a local daemon over
a unix socket had no option between "blocked" and allowAllUnixSockets, which
also exposes docker.sock, tmux and anything else listening on the machine.
seccomp-bpf cannot read a socket path out of user memory, so the allowlist is
enforced one level up. When any --allow-unix-connect entry is passed,
apply-seccomp installs a user-notification filter instead of the blocking one
and a supervisor process performs each connect()/bind()/listen() itself on the
caller's socket, permitting a unix connect only when the canonical target is
inside the allowlist.
The supervisor never answers SECCOMP_USER_NOTIF_FLAG_CONTINUE for those
syscalls: continuing re-executes the syscall from the caller's own memory and
file descriptor table, which a sibling thread can rewrite in between. Instead
it takes its own reference to the socket (pidfd_getfd), its own copy of the
sockaddr (process_vm_readv, with the notification id as the pid-reuse guard),
and connects through a pinned O_PATH handle of the target inode rather than
walking the path a second time, so no symlink or rename swap after the check
can redirect it. A 6-second stress run of 300k connects with a thread
rewriting the sockaddr, and another with the fd also being dup2-swapped
between AF_INET and AF_UNIX, land nothing on the forbidden socket; the same
harness against a filter that allows everything lands ~40k, so it bites.
Still refused with an allowlist configured:
- unix bind() and listen() - a sandboxed command connects to services, it
does not publish them. listen() is refused separately because the kernel
autobinds an abstract name behind bind()'s back when SO_PASSCRED is set
and a connect fails.
- the abstract namespace, which has no path to match against
- datagram unix sockets, including the SOCK_RAW spelling the kernel maps
onto SOCK_DGRAM, whose sendto() carries a destination path of its own
- seccomp(..., SECCOMP_FILTER_FLAG_NEW_LISTENER) and the seccomp
notification ioctls. Notifications go to the most recently installed
listener, so without this a workload could install one, receive its own
connect() traps and answer CONTINUE; denying it is also what makes a dead
supervisor fail closed (ENOSYS) rather than open.
The supervisor is forked before the PID-namespace unshare: after that unshare
a fork would land inside the sandbox's own PID namespace and the kernel
refuses CLONE_THREAD outright, leaving no way to service a blocking connect
off the receive loop. It is therefore neither addressable nor signallable
from inside the sandbox, sets PR_SET_DUMPABLE=0 so nothing can ptrace the
listener fd out of it, and caps brokered calls in flight so a workload cannot
exhaust it with parked connects.
Entries that do not resolve are dropped, and so are entries the sandbox can
also write: link(2) works on socket inodes, so a writable allow-listed
directory is one hard link away from every other socket the user owns. Give
the socket's directory a denyWrite carve-out to allow it.
Requires Linux 5.6 (pidfd_getfd). Older kernels print a warning and keep
blocking unix sockets entirely. With no allowlist configured the filter and
the process layout are unchanged, and TCP is unaffected in both modes.
The allowlist drops entries the sandbox can also write, because link(2)
works on socket inodes: a writable allow-listed directory is one hard link
away from every other socket the user owns. That check quietly did nothing
in the two cases where it matters most.
- No writeConfig at all. Nothing bounds writes, so every path is writable,
but the check saw an empty set of write roots and concluded nothing was.
- filesystem.disabled, which resolves to allowOnly: ['/']. Canonicalizing
stripped the trailing slash from "/" and left the empty string, which
then failed the absolute-path test and was discarded — so again no write
roots, and every entry passed through.
Both now drop every entry with a warning that says which configuration
caused it, and "/" survives canonicalization as the root it is.
The broker is sound only because it never answers SECCOMP_USER_NOTIF_FLAG_CONTINUE for connect(): the kernel would re-read the caller's memory and file descriptor table after the check, and a sibling thread can change both. That property was argued in comments and verified by hand; nothing in the build enforced it, so a later "simplification" of the supervisor into inspect-then-continue would have looked fine. test/fixtures/uds-race.c is that sibling thread. It runs in two shapes — one rewriting the shared sockaddr between an allowed and a forbidden path, one also flipping the fd number between an AF_INET and an AF_UNIX socket — and the verdict is external: the forbidden listener must accept nothing. The tests assert non-vacuity as well as the property, since "nothing reached the forbidden socket" is trivially true of a run that never connected: they require thousands of iterations and hundreds of successful connects. Checked against a deliberately unsound build that answers CONTINUE, where the forbidden listener accepts ~1,400 connections in two seconds and both tests fail. Also corrects the overhead figure in the README, which was carried over from a prototype rather than measured here (~8 us to ~57 us per TCP connect on arm64), and documents that the supervisor's death takes TCP down with it.
sylvesterkaczmarek
left a comment
There was a problem hiding this comment.
broker_thread() still has an allocation-failure path that can wedge the sandbox: resp = calloc(...) is nullable, but broker_handle() runs regardless. If that connect succeeds, the duplicated socket is already mutated, then no SECCOMP_IOCTL_NOTIF_SEND occurs and the original syscall remains blocked indefinitely. Could the response buffer be allocated before brokering (or otherwise guarantee a reply) and avoid executing the syscall when that allocation fails?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
network.allowUnixSocketsworks on macOS only. On Linux the seccomp filter deniessocket(AF_UNIX, ...)outright, so a command that talks to a local daemon over a unix socket has nothing between "blocked" andallowAllUnixSockets: true— which also hands it docker.sock, tmux, and anything else listening on the machine. This ports the setting to Linux.How
seccomp-bpf cannot read a socket path out of user memory, so the allowlist is enforced one level up. With any entry configured,
apply-seccompinstalls a user-notification filter instead of the blocking one, and a supervisor process performs eachconnect()/bind()/listen()itself on the caller's socket, permitting a unix connect only when the canonical target is inside the allowlist.It never answers
SECCOMP_USER_NOTIF_FLAG_CONTINUEfor those syscalls. Continuing re-executes the syscall from the caller's own memory and fd table, which a sibling thread can rewrite in between — inspect-then-continue is racy by construction. Instead the supervisor takes its own reference to the socket (pidfd_getfd, which shares the open file description, soO_NONBLOCK/EINPROGRESSstay the caller's), its own copy of the sockaddr (process_vm_readv, with the notification id as the pid-reuse guard), and connects through a pinnedO_PATHhandle of the target inode rather than walking the path again, so no symlink or rename swap after the check can redirect it.Still refused with an allowlist configured:
bind()andlisten()— a sandboxed command connects to services, it does not publish them.listen()is refused separately because the kernel autobinds an abstract name behindbind()'s back whenSO_PASSCREDis set and a connect fails.SOCK_RAWspelling the kernel maps ontoSOCK_DGRAM, whosesendto()carries a destination path of its ownseccomp(..., SECCOMP_FILTER_FLAG_NEW_LISTENER)and the seccomp notification ioctls — notifications go to the most recently installed listener, so without this a workload could install one, receive its ownconnect()traps and answer CONTINUE. Denying it is also what makes a dead supervisor fail closed (ENOSYS) rather than open.The supervisor is forked before the PID-namespace unshare: after that unshare a fork lands inside the sandbox's own PID namespace and the kernel refuses
CLONE_THREADoutright, leaving no way to service a blocking connect off the receive loop. So it is neither addressable nor signallable from inside the sandbox, setsPR_SET_DUMPABLE=0so nothing can ptrace the listener fd out of it, and caps brokered calls in flight so a workload cannot exhaust it with parked connects.Entries that do not resolve are dropped, and so are entries the sandbox can also write:
link(2)works on socket inodes andfs.protected_hardlinksonly stops linking another user's, so a writable allow-listed directory is one hard link away from every other socket the user owns. AdenyWritecarve-out on the socket's directory allows it.Where the sandbox restricts writes nowhere — no
writeConfig, orfilesystem.disabled(which isallowOnly: ['/']) — that rule has nothing to hold onto, so every entry is dropped with a warning naming the reason. Worth knowing for anyone force-enrolled into the sandbox without filesystem isolation: they get the old blocking behavior rather than an allowlist that cannot mean anything.Compatibility
pidfd_getfd). Older kernels print a warning and keep blocking unix sockets entirely — degraded, never permissive.SO_PEERCRED(uid/gid are the workload's), so servers that authorize by peer pid will not recognize the caller.Verification
Driven end-to-end through the
srtCLI on Linux, A/B againstorigin/mainwith the same config and fixture:origin/mainRace coverage: 6 s × 300k connects with a thread rewriting the sockaddr, and another run with the fd also
dup2-swapped between AF_INET and AF_UNIX — the forbidden listener accepted 0, self-detected violations 0. The same harness pointed at a filter that allows everything lands ~40k accepts, so it does detect what it claims to.New
test/sandbox/unix-socket-allowlist.test.ts(19 tests, Linux-only) covers allow/deny, symlinks both directions, relative paths, abstract, bind/listen, dgram + SOCK_RAW pairs, the NEW_LISTENER deny, TCP staying intact, dropped entries, grandchildren, and brokering alongside violation observation (both share the one listener the kernel allows). Cross-compiled clean for x86-64 and arm64 with-Wall -Wextra.