Skip to content

run: forward per-process UDP over IPv6 - #786

Merged
magurotuna merged 4 commits into
mainfrom
fix/ipv6-udp-forwarding
Jul 20, 2026
Merged

run: forward per-process UDP over IPv6#786
magurotuna merged 4 commits into
mainfrom
fix/ipv6-udp-forwarding

Conversation

@magurotuna

Copy link
Copy Markdown
Member

Summary

Fixes #781.

  • route Linux per-process IPv4 and IPv6 UDP through gVisor's UDP forwarder
  • preserve datagram boundaries while relaying each full-tuple flow through the selected daemon transport
  • keep transport dials off the TUN ingress loop and bound each session to 256 active UDP flows with pooled relay buffers
  • generate valid IPv6 UDP responses through gVisor and retain the scoped fd78::/64 child route
  • expire idle flows and close active relays when the daemon session ends

Verification

  • go test ./cmd/clawpatrol -run '^TestRunStackUDPForwarder' -count=1 -v
  • go test -race ./cmd/clawpatrol -run '^TestRunStack(TCP|UDP)Forwarder' -count=1
  • go test ./cmd/clawpatrol -run '^TestChildNetnsSteps$' -count=1 -v
  • go vet ./cmd/clawpatrol
  • make lint
  • gofmt and git diff --check

The package-wide suite still encounters the same host/network/plugin-dependent failures seen on unchanged 9edbeb0; the issue-focused tests are green.

@github-actions

github-actions Bot commented Jul 19, 2026

Copy link
Copy Markdown

Preview removed (PR closed).

@magurotuna
magurotuna force-pushed the fix/ipv6-udp-forwarding branch from df77db9 to 9a4f30b Compare July 19, 2026 09:17

@avocet-bot avocet-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@magurotuna

Copy link
Copy Markdown
Member Author

Ubuntu 24.04 QEMU E2E verification

I ran a live end-to-end verification against the exact PR head:

  • PR head: 9a4f30b04dd7049fdf80231ff871eb3ff5e59aa7
  • Base: 46d7c82b7c7b52973863821a627afd511dc1673f
  • PR run: forward per-process UDP over IPv6 #786 head build SHA-256: 220f2c8bb61f3699c7dd6949ff386994d1a2a209c909b4ae292e0cf306f303d1
  • Environment: Ubuntu 24.04.4 LTS, kernel 6.8, x86_64 QEMU/KVM
  • Execution path: non-root clawpatrol run, fresh disposable identity and VM overlay

UDP results: PASS

The new UDP paths worked as expected:

  • IPv4 default route remained default dev wg0.
  • IPv6 remained scoped to fd78::/64; no IPv6 default route was added.
  • IPv6 UDP/53 completed 3/3 queries successfully.
    • strace confirmed AF_INET6 / SOCK_DGRAM traffic and response receipt.
  • Generic non-DNS UDP over tsnet completed 2 flows × 3 datagrams: 6/6 responses.
  • Concurrent IPv6 DNS and generic UDP traffic passed.
  • The same probes passed after restarting the per-user daemon.
  • A disposable local WireGuard gateway fixture also completed 2 flows × 3 datagrams: 6/6 responses, both before and after daemon restart.
  • I found no panic, negative-refcount, invalid-memory, or use-after-free signature in the daemon logs.

Blocking TCP passthrough regression

The PR #786 head build consistently timed out on a public HTTPS passthrough probe. I repeated the same probe on the same VM and network with the exact base build and a diagnostic build based on the PR #786 head:

PR #786 head build:  timed out during TLS connection (curl exit code 28); no HTTP response
exact base build:    HTTP 200; TLS verification succeeded
diagnostic PR build: HTTP 200; TLS verification succeeded

The diagnostic build was based on the exact PR #786 head and differed only by the following one-line change in injectRunTunPacket:

diff --git a/cmd/clawpatrol/daemon_session_linux.go b/cmd/clawpatrol/daemon_session_linux.go
--- a/cmd/clawpatrol/daemon_session_linux.go
+++ b/cmd/clawpatrol/daemon_session_linux.go
@@ -133,7 +133,6 @@
 // does not consume the caller's reference, so every dispatch and drop path
 // releases it here.
 func injectRunTunPacket(ep *channel.Endpoint, version byte, pkt *stack.PacketBuffer) {
-	defer pkt.DecRef()
 	switch version {
 	case 4:
 		ep.InjectInbound(header.IPv4ProtocolNumber, pkt)

Diagnostic build SHA-256:

f28cd80d2201c7ca50d1ab1fe4fcd3b3e7e07e2478918e38297c68291eb9e32f

This A/B/A result strongly isolates the TCP regression to the new inbound PacketBuffer ownership/release behavior. I do not want to overstate the exact cause before confirming the ownership contract of the pinned gVisor version, but removing that release restored the same HTTPS probe.

A configured PostgreSQL native flow succeeded separately, so this is not a claim that every TCP path is broken. However, public TCP/HTTPS passthrough and an explicit IPv6 TCP connect both failed with the PR #786 head build.

Verdict

I recommend not merging this revision yet. The IPv6 and generic UDP implementation passed live E2E over both tsnet and WireGuard, but the TCP passthrough regression should be fixed first.

Suggested regression coverage:

  1. IPv4 public TCP/HTTPS passthrough.
  2. IPv6 TCP connect.
  3. Multiple datagrams per IPv4 and IPv6 UDP flow.
  4. TCP and UDP probes after daemon restart.

All disposable VM, gateway, client, identity, route, and temporary test artifacts were cleaned up after the run.

Ensure upstream UDP reads cannot outlive the shared flow idle lifetime. Re-check opposite-direction activity when a read deadline races a refresh, and cover the bounded deadline with a regression test.
Use errors.As when checking net.Error so the idle-deadline retry also recognizes wrapped timeout errors and satisfies errorlint.
@magurotuna

Copy link
Copy Markdown
Member Author

Follow-up: controlled root-cause correction and final verification

The initial candidate-vs-base HTTPS A/B result was not stable enough to attribute a TCP regression to this PR. A later same-identity/same-destination control showed exact base timing out in the same way.

An instrumented TUN/TCP/relay trace localized the current failure beyond the PR code:

  • child SYN reached gVisor;
  • transport Dial("tcp", ...) completed;
  • the gVisor endpoint was established;
  • the 517-byte TLS ClientHello was read and written 517/517 bytes to the tsnet remote connection;
  • the remote connection returned zero bytes until timeout.

In the same tsnet fixture, plain TCP/HTTP, IPv4 UDP DNS, and IPv6 UDP DNS all passed. All HTTPS destinations stalled, including exact base, while host-direct HTTPS passed. The current port-443 failure is therefore an external gateway/TLS fixture issue, not evidence of a candidate TCP regression. The speculative split-stack remediation was reverted because it did not address this failure and introduced ICMP/PMTU routing concerns.

The real lifecycle issue found during review was an unbounded upstream UDP Read. Commits 7416c0b and 78c971f now:

  • bound upstream reads by the remaining shared idle lifetime;
  • retry when opposite-direction activity races a read timeout;
  • handle wrapped net.Error values with errors.As;
  • add a RED-to-GREEN regression test that verifies the deadline is present and bounded.

Verification

  • independent review: APPROVE, no blocking findings;
  • focused forwarder tests: 100 repetitions passed;
  • race tests: 20 repetitions passed;
  • exact make lint: 0 Go issues, 0 dashboard warnings/errors;
  • QEMU/WireGuard: concurrent IPv4 + IPv6 UDP (20 rounds each), followed by TLS 1.3 / 188416-byte transfer: passed before and after daemon restart;
  • production 120-second idle expiry followed by same-tuple recreation: passed;
  • QEMU/tsnet: IPv4 UDP DNS, IPv6 UDP DNS, and plain TCP/HTTP: passed;
  • final daemon logs: no panic, negative refcount, UAF, data race, or leak signatures;
  • CI at 78c971f60c5394d5c3be39abf79c6efbea41ff39: 5/5 checks passed.

Cleanup completed: disposable identities/state, VM, temporary worktrees, SSH key, and QEMU overlay were removed. Full evidence is archived locally with SHA-256 96676610f12833c9175949fd2103ce7b81d205c220b6c7409b3a763fbf46ac85.

@magurotuna

Copy link
Copy Markdown
Member Author

Final-head UDP E2E addendum

I completed the remaining minimal UDP matrix against the exact final head:

  • Commit: 78c971f60c5394d5c3be39abf79c6efbea41ff39
  • Binary SHA-256: 55c3678f5416f6537c665b7f4b1ae8756826e87b091a2e783e86bad292c4264c
  • Environment: fresh Ubuntu 24.04.4 QEMU/KVM guest with a disposable guest-local Headscale control plane

Results:

  • tsnet before cold restart:
    • IPv6 DNS-VIP UDP/53: 3/3 passed
    • generic UDP/STUN: 6/6 passed across two sockets
  • tsnet after cold restart:
    • IPv6 DNS-VIP UDP/53: 3/3 passed
    • generic UDP/STUN: 6/6 passed across two sockets
  • WireGuard before cold restart:
    • generic UDP/STUN: 6/6 passed across two sockets
  • WireGuard after cold restart:
    • generic UDP/STUN: 6/6 passed across two sockets

The old daemon PIDs were stopped and replaced with new PIDs for both restart checks. Every tested daemon and the Gateway had an executable SHA matching the candidate binary.

All six IPv6 DNS straces used AF_INET6 and SOCK_DGRAM. The wrapped child retained an IPv4 default route through wg0, an IPv6 route only for fd78::/64, and no IPv6 default route.

Cleanup completed. Guest/host route snapshots and trust-store hashes were unchanged, fixture listeners were closed, and no credentials or private keys were retained.

Scope note: this run validates the exact final head using a local Headscale control plane. It does not add exact-base UDP evidence or validate normal Tailscale SaaS/dashboard onboarding.

Combined with the earlier exact-base/final-candidate TCP/443 Phase 1 result, the requested final-head TCP/UDP verification is now complete.

@avocet-bot avocet-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — re-reviewed the idle-read fixes and the final merge commit; CI is green.

@magurotuna
magurotuna merged commit 438b1bc into main Jul 20, 2026
5 checks passed
@magurotuna
magurotuna deleted the fix/ipv6-udp-forwarding branch July 21, 2026 03:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

clawpatrol run: support IPv6 UDP in the per-process TUN bridge

2 participants