Skip to content

feat: refresh circuit relay client reservations before expiry - #272

Merged
adust09 merged 2 commits into
mainfrom
feat/268-reservation-refresh
Aug 23, 2026
Merged

feat: refresh circuit relay client reservations before expiry#272
adust09 merged 2 commits into
mainfrom
feat/268-reservation-refresh

Conversation

@adust09

@adust09 adust09 commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Spec

specs/relay/circuit-v2.md, Reservation section:

the expire field contains the expiration time as a UTC UNIX time in seconds. The reservation becomes invalid after this time and it's the responsibility of the client to refresh.

the reservation remains valid until its expiration, as long as there is an active connection from the peer to the relay ... if the peer disconnects, the reservation is no longer valid.

Fixes #268 (depends on #266, already merged).

What changed

listenCircuit (src/LibP2P/NAT/Relay/Transport.hs) previously performed one RESERVE, ignored rsvExpire, and never refreshed — a node would silently become unreachable via the relay once its reservation lapsed.

  • Parses rsvExpire from the RESERVE response. If the relay omits it, the RESERVE (initial or refresh) is treated as a failure — there's no expiry to schedule a refresh against, so trusting an undated reservation would defeat the whole point of this change.
  • A background refreshLoop, spawned once the initial reservation is granted, polls (rrcPollInterval) and re-issues RESERVE on a fresh hop stream over the same connection to the relay once the reservation is within rrcMargin of expiry.
  • A disconnect notifier (reusing the swDisconnectNotifiers hook from fix: invalidate relay reservations when the peer's last connection closes #271) matches the specific connection used for the reservation (by connState TVar identity, not peer id) and immediately withdraws the circuit listen address if that connection closes.
  • A refresh that fails (transport error, or a non-OK STATUS) also withdraws the listen address.
  • New primitive switchWithdrawListener (src/LibP2P/Switch/Listen.hs) removes one ActiveListener from the Switch — cancels its accept loop, closes it, and drops it from swListeners — without touching any other listener or connection. switchClose had equivalent inline logic for tearing down all listeners together; nothing previously let a single listener stop advertising on its own.

circuitTransport and listenCircuit now take a ReservationRefreshConfig; NATConfig gained ncReservationRefresh (default via defaultReservationRefreshConfig) so registerNATHandlers threads it through. Both are re-exported from the LibP2P facade alongside the existing circuitTransport/CircuitState exports.

go-libp2p comparison

Checked p2p/host/autorelay/relay_finder.go (the refresh loop lives there, not in p2p/protocol/circuitv2/client/reservation.go, which only does a single RESERVE):

  • rsvpExpirationSlack = 2 * time.Minute, rsvpRefreshInterval = time.Minute — refresh once expiry is within 2 minutes, checked every minute. Mirrored exactly as defaultReservationRefreshConfig (rrcMargin = 120, rrcPollInterval = 1 minute in µs).
  • On refresh failure, go-libp2p drops the relay from its active set and unprotects the connection in the connection manager, but does not force-close the connection. Mirrored: switchWithdrawListener withdraws the listen address but never calls closeConnection on the relay connection itself.
  • reservation.go's Reserve treats an expiration in the past (which includes a zero/absent expire, since protobuf-optional fields decode to the zero value) as a hard error. Mirrored via reservationExpiry, which fails outright — for both the initial RESERVE and a refresh — when rsvExpire is missing, rather than defaulting or best-effort-continuing.

One divergence: go-libp2p's relay-loss handling flows through its general connection-manager "unprotect" mechanism, which is polling/eviction-based within its connmgr, not an immediate synchronous callback. This codebase already has a synchronous swDisconnectNotifiers hook (added in #271 for the server-side reservation cleanup), so the client side uses the same mechanism for symmetry and to get immediate (not next-poll) withdrawal on disconnect.

Design decisions

  • Peer-keyed loss detection, matching the server side. The disconnect notifier withdraws the circuit listen address only once lookupConn finds no connection left to the relay peer, mirroring registerReservationCleanup in NAT.hs.

    The two reference implementations each pick one model and apply it on both sides. go-libp2p is peer-keyed throughout: the relay server returns early from its disconnect notifiee while Connectedness(p) == Connected (circuitv2/relay/relay.go), and the client's relay_finder skips the event unless Connectedness == NotConnected before deleting from its map[peer.ID]*Reservation (autorelay/relay_finder.go). rust-libp2p is connection-keyed throughout, on the server (HashMap<PeerId, HashMap<ConnectionId, Reservation>>) and on the client (reservation_addresses keyed by ConnectionId).

    This repo adopted the go-libp2p model on the server side in relay: invalidate reservations when their owning connection disconnects #255, so the client matches it. Mixing the two also misbehaves against a real go-libp2p relay, which keeps a reservation while any connection from us remains: per-connection matching would drop a listen address that is still routable.

What was tested

New test/LibP2P/NAT/Relay/ReservationRefreshSpec.hs, three real in-process switches over loopback TCP, following the existing TransportSpec.hs / ReservationLifecycleSpec.hs conventions:

  1. Refresh keeps the client reachable past the original expiry. Relay configured with a 2s reservation duration and an aggressive refresh config (1s margin, 100ms poll); after waiting past the original expiry, a fresh dial + ping through the relay still succeeds. This one necessarily uses real wall-clock waiting (~4s total) since it's exercising the actual background timer, but the interval is configurable specifically so it doesn't have to wait anywhere near go-libp2p's real defaults (2 minute margin).
  2. Connection loss withdraws the listen address immediately, driven deterministically via closeConnection (no polling), following ReservationLifecycleSpec's established pattern for synchronous notifier-driven assertions.
  3. A relay that refuses a refresh withdraws the listen address, without affecting other connections or listeners. Uses a hand-rolled fake relay (custom hop protocol stream handler) that grants exactly one RESERVE then refuses every subsequent one, since the real relay server here always grants a refresh from an existing holder even at capacity. Confirms an unrelated plain-TCP connection into the same target Switch survives (ping still round-trips) and that Switch's other listener is untouched.

Also updated TransportSpec.hs and ReservationLifecycleSpec.hs for the new circuitTransport/NATConfig signatures (no behavioral changes to those tests).

Full suite: 1153 examples, 0 failures (was 1150 baseline + 3 new).

What I deliberately left out

  • No enforcement of RelayLimit/_mLimit (issue relay: enforce client-side circuit limits (data/duration) #269) — out of scope per the brief, untouched.
  • No exponential backoff on repeated refresh failures before withdrawing — go-libp2p doesn't retry either; it withdraws on the first failure. A future improvement, not attempted here.
  • I did not verify behavior against an actual go-libp2p relay (no interop harness run) — this is pure in-process Haskell-to-Haskell testing. The go-libp2p comparison above is from reading its source, not from a live interop test.
  • I did not test what happens if listenCircuit is called twice for the same relay (the second registration overwrites the first's InboundQueue in CircuitState); this is a pre-existing property of registerQueue, not something this change introduces or was asked to address.

specs/relay/circuit-v2 makes reservation refresh the client's
responsibility ("the reservation becomes invalid after this time and
it's the responsibility of the client to refresh") and ties validity
to the connection to the relay ("if the peer disconnects, the
reservation is no longer valid"). listenCircuit previously performed a
single RESERVE and ignored rsvExpire, so a node would silently become
unreachable via the relay once its reservation lapsed.

listenCircuit now tracks the granted expiry and runs a background loop
(ReservationRefreshConfig, tuned like go-libp2p's autorelay finder:
2 minute margin, checked every minute) that re-issues RESERVE on the
existing hop connection ahead of expiry. A disconnect notifier
withdraws the circuit listen address immediately if that specific
connection to the relay is lost, and a failed refresh (transport error
or non-OK STATUS) withdraws it too, so a dead reservation stops being
advertised in switchListenAddrs instead of dialers hitting a live-looking
but dead address.

Withdrawal is a new general primitive, switchWithdrawListener, that
removes one ActiveListener from the Switch without touching the rest of
its listeners or connections -- switchClose already had inline logic to
tear down all listeners together, but nothing let a single listener stop
advertising on its own before this.
The disconnect notifier matched the specific connection the RESERVE went
out on, so a client with a second connection to the same relay withdrew
its circuit listen address while the relay still honoured the
reservation.

The two reference implementations each pick one model and apply it on
both sides. go-libp2p is peer-keyed throughout: the relay server returns
early from its disconnect notifiee while Connectedness(p) == Connected
(circuitv2/relay/relay.go), and the client's relay_finder skips the event
unless Connectedness == NotConnected before deleting from its
map[peer.ID]*Reservation (autorelay/relay_finder.go). rust-libp2p is
connection-keyed throughout, on the server (HashMap<PeerId,
HashMap<ConnectionId, Reservation>>) and on the client
(reservation_addresses keyed by ConnectionId).

This repo adopted the go-libp2p model on the server side in #255, so the
client has to match it. Mixing the two also breaks against a real
go-libp2p relay, which keeps a reservation while any connection from us
remains: per-connection matching drops a listen address that is still
routable.

Withdraw the listener only when lookupConn finds no connection left to
the relay peer, mirroring registerReservationCleanup.
@adust09
adust09 merged commit ca9b70b into main Aug 23, 2026
3 checks passed
@adust09
adust09 deleted the feat/268-reservation-refresh branch August 23, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

relay: refresh client reservations before expiry

1 participant