chore: merge upstream irlserver/srtla_send c9f6bb2 (0 behind) + selective port set - #24
Merged
Conversation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Kalman filter tracks RTT velocity but congestion control ignored it. Now perform_window_recovery() halves the recovery rate when velocity exceeds 2.0 ms/sample, preventing window inflation during active congestion. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a velocity_penalty term to predicted_arrival() when Kalman velocity is positive. This penalises links with rising RTT trends before congestion manifests as loss, giving EDPF proactive avoidance. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Exclude links where in-flight bytes exceed 1.5× the bandwidth-delay product (BDP). Prevents runaway in-flight during RTT inflation on cellular networks. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add LinkPhase enum (Registering → Warming → Live → Degraded → Cooldown) to replace implicit state from boolean flags. Scheduler skips non-Live/ Degraded links, eliminating early NAK bursts from newly-connected links. - Warming phase requires 2 RTT probes or 5s timeout before going Live - Housekeeping drives degradation detection and cooldown transitions - All selection strategies (classic, enhanced, RTT-threshold, EDPF, BLEST) now check is_schedulable() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add AsymmetricEwma struct with separate alpha_up / alpha_down smoothing factors. Will be used to replace ad-hoc asymmetric smoothing in capacity and RTT tracking. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add optional TOML configuration file support: - New --config CLI arg for specifying config file path - TomlConfig struct with serde defaults for all tunable constants (congestion control, EDPF scheduler, link lifecycle, selection) - Load at startup with fallback to defaults on error Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Detect keyframe bursts (runs of max-MTU 1316-byte packets) and prefer higher-quality links for keyframe packets. The KeyframeDetector tracks consecutive max-size packets and declares a burst after 5+ in a row. During keyframe bursts, the scheduler selects the link with the highest quality_multiplier among connected/schedulable links, ensuring keyframes travel on the most reliable path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implement RFC 8382 statistical shared bottleneck detection using per-link OWD samples from Kalman RTT/2. Each interval computes: - Skew (mean−median): queuing delay buildup - Variance (MAD/mean): delay variability - Frequency (sign-change ratio): oscillation pattern - Loss rate from NAK counts Links are bottlenecked when skew > C_S AND (var > C_H OR loss > P_L), then grouped by delay statistics similarity using union-find. In EDPF mode, correlated links have effective capacity reduced by 0.7x. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… items Gate test-only methods behind #[cfg(test)] instead of suppressing dead_code warnings: AsymmetricEwma, edpf::select_from/select_from_indices, sbd::groups(), keyframe::is_in_burst/total_bursts, blest::record_blocking, iods::reset. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Augments the packet-size keyframe heuristic with an out-of-band hint
channel. An upstream encoder (belacoder) can tell srtla_send "the next
N SRT data packets are critical" via the Unix control socket, e.g.
mark-critical 23
Maintains a per-sender budget in DynamicConfig; each data packet
consumes one from the budget. While budget > 0 or the heuristic fires,
the scheduler routes to the highest-quality link. The two signals are
OR-combined so heuristic-only deployments (ffmpeg, libsrt) keep the
existing behaviour while hint-aware encoders gain exact coverage of
IDR / SPS / PPS frames that the 5-packet-burst heuristic misses.
The previous flat text protocol ("mode classic", "mark-critical 3",
"status") was shaped for humans poking with socat. Machine clients
(belacoder, future agents, exporters) need request/response correlation,
typed errors, and schema-discoverable methods.
One message per line, one JSON-RPC 2.0 envelope per message. Requests
without an id are notifications, used for the hot-path mark_critical
hint so encoders don't block on a round-trip when firing a keyframe.
Methods: set_mode, set_quality, set_exploration, set_rtt_delta,
get_status, get_stats, mark_critical. subscribe / unsubscribe names
are reserved for a later streaming upgrade.
Hand-rolled rather than pulling in jsonrpsee or jsonrpc-core: the spec
is one page, srtla_send's control listener is a blocking std::thread
(not tokio), and deps are intentionally lean here. Full protocol docs
in docs/CONTROL_PROTOCOL.md.
BREAKING: the old "mode classic" / "stats" / etc. text commands no
longer work. Update any scripts poking the control socket.
The JSON-RPC mark_critical method had three fragility issues: the packet count was a guess (MPEG-TS overhead unknowable from encoder side), the hint went on a different scheduler path than the SRT data so it could arrive after the packets it described, and in multi-source setups each encoder's hints polluted the shared counter. Swap to a dedicated UDP sidecar. The encoder sends a 5-byte datagram (magic byte + window_ms) that opens a deadline-based critical window. Loopback UDP shares the network stack with SRT data, so the window opens tightly ordered against the packets it protects. The scheduler OR-combines the window with the packet-size heuristic as before. Operator enables with --priority-bind ADDR:PORT. get_status exposes windows_received and malformed_datagrams counters. Full wire-format docs in docs/KEYFRAME_PRIORITY.md. BREAKING: mark_critical RPC removed from the JSON protocol. Encoders that wired the old hint-count API must switch to the UDP sidecar.
New optional HTTP endpoint for scraping. Exposes per-link series (up, RTT, window, in_flight, NAKs, bitrate, quality_multiplier), aggregate counters, scheduling mode as a gauge, and priority-sidecar counters. Enabled via --metrics-bind ADDR:PORT. Hand-rolled over tokio::net::TcpListener with no axum/hyper/tower pulled in. Supports GET /metrics and GET / only; anything else 404s. Responses always close the connection, which is plenty for Prometheus scrape semantics.
Scraping get_stats at 1 Hz loses sub-second link-state changes.
Subscriptions let a client register for a topic and receive server
push events as JSON-RPC notifications on the same connection.
Topics:
- stats — StatsSnapshot pushed once per second alongside the
existing housekeeping update
- priority.window — pushed on each accepted sidecar datagram with
at_ms, window_ms, deadline_ms
The sync std::thread Unix-socket listener couldn't push unsolicited
messages on the same connection. Replaced it with a tokio
UnixListener whose per-connection task tokio::selects between reads
and outbound push-channel writes. Stdin stays blocking — no
subscription support there, subscribe/unsubscribe from stdin returns
method not found.
SubscriptionHub fans out by topic into per-connection mpsc senders;
full channels drop events (backed-up subscriber never blocks the
producer) and closed channels are pruned lazily.
BREAKING: `config::spawn_config_listener` is gone; call
`config::spawn_stdin_listener` and `control_socket::spawn` separately.
drop the unproven scheduling modes (rtt-threshold, edpf) and their auxiliary filters (sbd, blest, iods). only the IRL-tested classic and enhanced modes remain. exploration stays as an off-by-default opt-in flag inside enhanced. removes: --rtt-delta-ms CLI flag, set_rtt_delta JSON-RPC, rtt_delta_ms config and stats fields, edpf_* TOML keys, rtt_threshold tests. removes ~1700 LOC. clears the deck before adding the weak-link classifier and per-link target-rate soft cap to enhanced mode.
new module sender/selection/classifier.rs implementing a three-tier delay cascade with entering/leaving hysteresis (3x ratio). classifies each connection as weak based on: - rtt vs the chosen tier (best=40%, safe=50%, max=60% of estimated budget; budget = max(longest_rtt*3, 500ms) capped at 5s), - bandwidth share vs an enter/leave threshold pair derived from fair share (0.25/N enter, 0.75/N leave). constants picked conservative for first soak; real-world observation may suggest retuning. shadow mode: classifier runs on the housekeeping tick and surfaces weak/reason/share/threshold per link plus selected_delay_ms / estimated_max_delay_ms via the existing get_stats json. selection is unchanged — admission gate wires in after a soak window.
new module sender/selection/link_cc.rs implementing a 3-state cc controller per connection (bootstrap / climbing / holding / backing_off) producing target_bps as a soft cap. inputs: - age-bucketed rtt ewma with 1:1 / 1:4 / 1:8 / 1:16 weights at age bands >=1s / >=500ms / >=250ms / <250ms; 2s gap snaps to the new sample. - rttvar via 1:3 weighted moving deviation. - 1s sliding-window loss permille (nak plumbing follow-up). - observed bps from existing BitrateTracker. state transitions: - loss > 5 permille (0.5%) -> BackingOff (multiplicative -15%). - rtt ewma > 1.5x rtt min -> Holding. - otherwise -> Climbing (additive +2% per tick, capped by 2x measured throughput so idle links don't ramp). shadow mode: snapshots flow into stats json next to existing weak/cc fields. selection is unchanged. wires into Enhanced as a soft cap after a soak window.
promote the weak-link classifier and per-link cc state from shadow mode into the enhanced selection scorer. - new fields weak / cc_backing_off on SrtlaConnection, stamped each housekeeping tick from WeakLinkFilter::classify and LinkCcController::tick_all. - enhanced::select_connection skips weak or backing-off connections when at least one healthy alternative is schedulable. when every link is weak, falls back to the full pool — better to send on a weak link than to drop the packet. regression tests cover the three branches: skip-weak-with-alternative, fallback-when-all-weak, and treats-backing-off-as-weak. 229 lib tests green.
extend the per-link CC state machine from 4 states (Bootstrap / Climbing / Holding / BackingOff) to 5 + a climb sub-mode. closes the gap between our simplified controller and the cellular profile's needs without porting the full 9-mode reference. new state: Drain — one-shot 25% multiplicative decrease when RTT inflation crosses 2.0x without observed loss. catches BDQ overload before ARQ surfaces it. transitions to Climbing on the next tick (or to Holding if RTT didn't recover). new sub-modes for Climbing: Hai — 6%/tick AI when RTT variance ≤ 10% of the smoothed RTT mean. confident headroom signal so we ramp faster. FastRecovery — 4%/tick AI for 5 ticks after exiting BackingOff or Drain. claws back the bandwidth we just gave up without the overshoot risk that Hai would carry. Normal — 2%/tick AI baseline; covers steady-state with no special signal. priority ordering when picking the climb sub-mode: 1. FastRecovery while the post-backoff window is open. 2. Hai when RTT is stable enough. 3. Normal otherwise. precedence in next-state selection: loss observed -> BackingOff rtt_inflation ≥ 2.0 -> Drain rtt_inflation > 1.5 -> Holding else -> Climbing new ClimbMode telemetry exported via LinkCcSnapshot and the per- link stats JSON (cc_climb_mode field). dashboards can render it alongside cc_state. 5 new unit tests: - hai_kicks_in_when_rtt_is_stable - hai_yields_to_normal_when_rtt_is_jittery - fast_recovery_engages_after_backoff - drain_triggers_on_high_rtt_inflation_no_loss - drain_then_recovery_path existing holding_when_rtt_inflates updated: original used 60ms samples (3x inflation) which now triggers Drain rather than Holding; switched to 35ms samples (1.75x — Holding band) so the test still exercises the original intent. 234 srtla_send lib tests pass.
batch_send.rs previously buffered up to a fixed 16 packets / 15ms flush window. on idle links that adds latency for traffic that arrives in bursts; on heavy links it caps how much we can amortise into a single sendmmsg-style call. three regimes, picked per connection from observed bitrate: bitrate regime batch threshold ───────────────── ─────────── ─────────────── ≤ 500 kbps LowActivity 4 500 kbps – 5 Mbps Normal 16 (Moblin sweet spot) > 5 Mbps HighLoad 32 flush interval stays 15ms across regimes — going longer would add latency on traffic resumption, going shorter would erase the syscall amortisation we batch for. new BatchRegime enum + BatchSender::set_regime / regime() accessor. SrtlaConnection grows a recompute_batch_regime() that maps current bitrate_bps → BatchRegime via BatchRegime::from_bps. housekeeping calls it once per tick alongside calculate_bitrate / update_phase. cheap to drive — set_regime is a single field write, no allocation. no-op when the regime hasn't changed (caller doesn't need to track deltas). 2 new unit tests: - regime_from_bps_thresholds — boundary semantics - batch_size_threshold_per_regime — actual flush trigger varies 236 srtla_send lib tests pass.
uplink sockets were always bound by source ip, which only steers egress on a multi-homed host with source-based routing. introduce an UplinkBinder trait so the steering action is injectable: SourceIpBinder keeps the cli behavior, CallbackBinder lets a library consumer steer the raw fd (android Network.bindSocket) while keeping IpAddr as the uplink identity. thread the binder through run_sender_with_config, connection creation, and reconnect.
collapse cooldown match guard, use clamp/iter/if-let/or_default in the selection code, drop the redundant LinkCongestionState::new, and allow constant-invariant assertions in the protocol test modules.
reconcile docs with the code after the edpf / rtt-threshold scheduling work was dropped. only classic and enhanced modes remain. - add CHANGELOG.md covering changes since v3.0.0, including an "explored and removed" section for the edpf/blest/iods pipeline - README: drop rtt-threshold mode, --rtt-delta-ms, and the stale set_rtt_delta / mark_critical control-socket examples; add the real --config, --priority-bind, --metrics-bind flags and the subscribe/unsubscribe methods - CONTROL_PROTOCOL: trim set_mode to classic/enhanced, remove the set_rtt_delta method and the rtt_delta_ms get_status field - remove docs/RTT_THRESHOLD_SCHEDULING.md
before this commit, LinkCongestionState::record_loss was only called from unit tests. the production sliding-window loss tracker stayed at zero permille, which meant CcState::BackingOff was unreachable in production and the cc_backing_off gate that enhanced selection now consults could never fire. CC's loss path was dormant. new LinkCongestionState::observe_traffic(bytes_sent_total, nak_total, now_ms) computes per-tick deltas against a previous-call baseline and forwards them to record_loss. first call stashes a baseline without sampling. byte delta is converted to a packet count using the standard SRT payload (1316 B); the ratio is invariant under uniform packet-size assumptions so the approximation is fine for the loss permille EWMA. LinkCcController::tick_all reads conn.bitrate.bytes_sent_total and conn.total_nak_count() and calls observe_traffic before the existing tick(). zero-traffic ticks are skipped so the window doesn't fill with no-op samples. quiet-link-with-NAK pathological case is bounded by synthesizing a single-packet "sent" baseline, so the ratio never divides by zero. removed the stale "follow-up commit" comment and dropped the #[allow(dead_code)] on record_loss now that it's wired. 3 new unit tests: - observe_traffic_first_call_sets_baseline_without_sample - observe_traffic_delta_flows_into_record_loss - observe_traffic_quiet_tick_with_naks_does_not_panic 239 srtla_send lib tests pass.
before this commit, cc_target_bps was computed per-tick by the CC
controller, surfaced via stats JSON, but never consumed in selection
— the stats comment even said "selection does not yet treat as a
soft cap. After a soak window the cap wires into the Enhanced score."
that wiring never happened.
new on SrtlaConnection:
pub(crate) cc_target_bps: u64,
stamped alongside cc_backing_off in sender/mod.rs from the per-tick
LinkCcSnapshot.
new in enhanced.rs:
fn cc_soft_cap_multiplier(conn) -> f64 in [CC_SOFT_CAP_FLOOR, 1.0]
formula:
headroom = max(0, cc_target_bps - measured_bps)
multiplier = clamp(headroom / cc_target_bps, FLOOR, 1.0)
short-circuits to 1.0 when:
- cc_target_bps == 0 (CC hasn't bootstrapped yet)
- measured_bps == 0 (idle link, plenty of headroom)
floor = 0.10 — saturated links keep 10% of their raw score so a
trickle of keepalive traffic still flows and the CC controller
keeps observing the link. without a floor, a link at exactly its
cap would get score 0 forever.
the multiplier folds into the link's existing quality-aware score:
score = base * quality_mult * cap_mult
same code path covers the non-quality branch:
score = base * cap_mult
the previous binary cc_backing_off gate still runs first as a hard
admission filter. the new multiplier is a soft signal that operates
within the surviving candidate pool — links upshifting close to
their CC ceiling get deprioritised before backoff fires.
4 new unit tests cover the multiplier helper:
- cap_no_signal_returns_unity
- cap_idle_link_returns_unity
- cap_at_target_falls_to_floor
- cap_half_target_returns_half
stats.rs comment updated to reflect production consumption. test
helpers default cc_target_bps to 0. 243 srtla_send lib tests pass.
batch_send.rs has tracked a per-connection BatchRegime (LowActivity / Normal / HighLoad) since the adaptive batch-send commit, driven each housekeeping tick from observed bitrate. it was never plumbed into the stats JSON though, so dashboards couldn't see why one link was batching more aggressively than another. new on LinkStats: pub batch_regime: String, populated from conn.batch_sender.regime().as_str().to_string() in SharedStats::update — same place cc_state and cc_climb_mode land. new BatchRegime re-export from connection::mod alongside BatchSender so external callers (stats, future telemetry) don't have to reach into batch_send.rs directly. format mirrors the existing cc_* string-field convention so the existing dashboard pattern that renders cc_state as a chip works for batch_regime with zero schema gymnastics.
…tent ported in follow-up commits per docs/notes/upstream-sync-2026-08-evaluation.md)
… hardening
The SRT control header is 16 bytes (type, type-specific info, timestamp,
destination socket id), but parse_srt_nak read the loss list from offset 4.
Every NAK therefore decoded three header words as lost sequence numbers while
dropping the real leading loss-list entries.
Add SRT_CONTROL_HEADER_LEN = 16 and read the loss list there; a frame shorter
than 20 bytes carries no loss list and yields an empty result.
Introduce SrtSeq, a newtype over the 31-bit sequence-number domain with
RFC1982-style modular comparison (serial_lt/le/gt/ge), a directional distance,
and a next() that wraps 0x7FFF_FFFF to 0. Range expansion now steps with
next() and breaks after emitting the end value, so a start==end==0x7FFF_FFFF
range emits exactly one entry instead of running the ring, and wrap-crossing
ranges expand correctly. Descending ranges, and the antipodal pair where
serial order is undefined, expand to nothing. A range end word with bit 31 set
is not a sequence number: that range is skipped and parsing continues.
parse_srt_nak now returns NakList { seqs, truncated }. The 1000-entry cap is
global across singles and range expansion, and hitting it stops parsing and
sets truncated so the loss report is not silently incomplete. The parser stays
pure; the caller in connection/packet_io.rs emits the warning, rate limited to
one per second per connection via SrtlaConnection::last_trunc_warn_ms.
Migrate every NAK fixture off the old 8-byte offset-4 assumption, using a
nonzero timestamp and socket id so a regression back to offset 4 would surface
those words instead of hiding in zeros. Add coverage for the ignored control
header, boundary sequence numbers, ranges at the domain maximum, wrap
crossing, descending and invalid-endpoint ranges, both sides of the truncation
boundary, and lengths 16..19. A dedicated test feeds the old offset-4 frame and
asserts it now decodes to nothing, documenting the deliberate break.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
…wedge Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
…la_send, not a hardcoded target/debug path Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
…refix-commit semantics
…ND_BIN override Measurement infrastructure for the switch-cooldown / flush-on-switch A/B evaluation of upstream 0cc0c6d and 24b5f64. Committed unconditionally: a rejected step still leaves reusable measurement tooling. - src/ab_metrics.rs: three process-lifetime monotonic AtomicU64 counters (switch / NAK / cooldown-hold), Arc-shared, never reset. Deliberately not sourced from SrtlaConnection::nak_count (reset by the congestion controller) or SharedStats (housekeeping-cadence snapshots). With test-internals off every entry point is an empty inline fn, so the shipped binary is unchanged. - config.rs: test-internals-gated `metrics` runtime command returning one JSON line, so a runner can difference the counters across an exact window. - Increment sites: the NAK loop (packet_handler), the Some(prev)->Some(new) connection switch (initial selection excluded), and both cooldown-suppression returns (enhanced + rtt-threshold). - tests/ab_switch_eval.rs: #[ignore]d paired runner over a real SRT path (srt-live-transmit caller -> srtla_send -> srtla_rec -> sink) with the frozen deterministic scenario (3 uplinks, fixed 20/60/120 ms delay, no loss or jitter, 4 Mbit/link, 6 Mbit offered, 5 s warm-up, 30 s window, 3 alternating paired runs per mode), plus a cheap single-run smoke test. - network-sim: wait_for_registered_uplinks exposed as a free function so stacks assembled outside SrtlaTestStack share one readiness definition; SrtlaTestStack delegates to it. - test_helpers: advance_test_clock is cfg(test) only. tokio::time::advance needs tokio/test-util, which arrives via dev-dependencies, so gating it on test-internals made `cargo build --features test-internals` fail to compile. Step A (remove flush-on-switch) measured REJECT under the pre-committed numeric adoption rule; Step B is therefore dependent-on-rejected and unmeasured. No production selection behavior changes in this commit.
… gate tests/netns_unconnected.rs was added alongside the sendmmsg/unconnected-socket work but never listed in scripts/netns_test_gate.sh. AGENTS.md directs operators to run privileged targets only through that bounded runner, so the new target had no bounded path and was exercised only by an unbounded --all-features run. Gate rerun: 7/7 targets executed and green, netns_unconnected 2 passed in 11.02s, well inside the 90s per-target cap.
…nnable `uv run scripts/workflow_authority_contract_test.py` is a documented gate command but had neither PEP-723 inline metadata nor a __main__ entrypoint: it exited 1 on ModuleNotFoundError for yaml, and once the dependency was declared it exited 0 having run zero tests. Its five assertions only ever executed indirectly, via the import in release_workflow_contract_test.py. Add the same inline metadata and unittest.main() entrypoint both sibling contract scripts already carry, and ignore the __pycache__/ directory those scripts produce so a gate run leaves the tree clean. Standalone run now reports 5 tests OK; release_workflow_contract_test.py still reports 13 tests OK.
…ool reorder Two gaps in the REG3 phase gate added by the sendmmsg/unconnected-sockets todo. handle_reg3 checked awaiting_reg3 membership but never consumed the grant on the success path (unlike handle_reg_err, which removes it). An index therefore stayed "awaiting" forever after its first successful REG3, so a duplicate or replayed REG3 -- a receiver retransmission, or a spoofed frame from any host that can reach the uplink's ephemeral port, which the accept-any source policy explicitly permits -- re-fired RegistrationEvent::Reg3 and made packet_io call clear_pre_registration_state() on a live, forwarding link, wiping packet_log, in_flight_packets, highest_acked_seq, congestion state and the batch queue. The grant is now removed on success, so a replay falls through to Reg3OutOfPhase (counted + ignored) and a legitimate reconnect re-arms it via send_reg2_to. SrtlaRegistrationManager is a single process-lifetime instance whose awaiting_reg3 / pending_reg2_idx / reg1_target_idx / probe_results are keyed by the positional index into the connections vector, but apply_connection_changes rebuilds that vector in ips-file order on SIGHUP without touching them. A stale grant could then authorize a REG3 on whichever uplink inherited the index. The reload now calls reset_index_scoped_state() whenever the pool order actually changed; surviving links keep their own SrtlaConnection::connected state, so only incomplete registration attempts are discarded and they retry. An unchanged reload still disturbs nothing. Tests: replayed_reg3_does_not_wipe_a_live_connection drives the real process_packet path and asserts the in-flight state is untouched by the replay; sighup_reorder_clears_stale_registration_index_state and sighup_unchanged_list_keeps_registration_index_state pin both halves of the reload behavior. Each fails on the pre-fix code.
Two rows in the 138-commit triage table were stale. b909220 (network-sim child-pipe drain) read REJECT / "not confirmed needed", but todo 8 verified the wedge against our own harness and implemented it as d08c517 (122 insertions in crates/network-sim/src/harness.rs, regression test child_pipes_are_drained_during_sustained_output). Now ADOPT-WITH-FORK-FIX -- fork-fix rather than a clean port because the drain threads are shaped for our harness rather than copied from upstream. 86b90aa (bind uplinks by interface index on apple targets) read ADOPT, but nothing was ever ported: git grep for apple / IP_BOUND_IF finds nothing under src/, and no todo in this plan had it in scope. Now DEFER-FOLLOWUP. Row count, PENDING-EVAL count and verdict-vocabulary completeness are unchanged (138 / 0 / 138); check-doc-refs.sh still exits 0.
…etry A partially failed REG2 broadcast retries on the next tick, but the retry resent to every uplink unconditionally — including ones already connected whose one-shot awaiting_reg3 grant handle_reg3 had consumed. The re-insert re-armed the gate, so a receiver-retransmitted REG3 could again wipe a live, forwarding uplink's packet log, in-flight count, and congestion state — the bug eaced59 fixed, reached through the broadcast retry. The retry now skips uplinks that are connected or already awaiting REG3. Also documents the registration hardening (eaced59 + this fix) in AGENTS.md/README.md per Rule A, corrects the triage doc's stale 'swap only on success' DNS wording to the detect-only design todo 12 actually shipped (85f5544) plus a DEFER-FOLLOWUP note for coordinated whole-bond receiver migration, and adds the missing todo-7 row to todo 14's acceptance matrix.
…iled REG2 A REG_ERR was acted on unconditionally: `handle_reg_err` ran for any index and `packet_io.rs` turned every one into `connected = false` on the receiving link. Uplink sockets are deliberately unconnected and SRTLA control frames carry no authentication, so a forged 2-byte REG_ERR from anything able to reach an uplink's ephemeral port force-disconnected an established, actively-forwarding link — and, because the handler cleared the global `pending_reg2_idx` / `reg1_target_idx` / `pending_timeout_at_ms`, it also aborted an unrelated uplink's concurrent handshake. REG_ERR is now gated exactly like REG3: honored only when the index is genuinely mid-registration (`pending_reg2_idx == Some(idx)` or an `awaiting_reg3` member). Rejected frames increment `out_of_phase_reg_err` and return the new `RegistrationEvent::RegErrOutOfPhase`, a no-op for the caller. In-phase clearing is scoped to the state that index actually owns. `send_reg2_to` additionally revokes any pre-existing `awaiting_reg3` entry when the send fails, so a failed resend cannot leave a REG3 authorization alive for a socket generation that was never re-armed. Pinned by four new tests in `src/tests/batch_io_tests.rs`; the three defect tests were proven to fail against a scratch mutation restoring the old behavior.
…n state A REG_NGP restarts the handshake at REG1, which re-arms `pending_reg2_idx` — exactly the state the REG_ERR phase gate treats as in-phase. `handle_reg_ngp` accepted one on `active_connections == 0 && pending_reg2_idx.is_none()`, so a forged REG_NGP followed by a REG_ERR could still tear down an established uplink, routing around the REG_ERR gate through a legal-looking transition. `active_connections` is a manager-side counter recomputed only by a housekeeping tick, while `SrtlaConnection::connected` is set immediately in the REG3 dispatch, so a genuinely connected link briefly reads zero. `handle_reg3` has also already consumed that index's `awaiting_reg3` entry by then, so a grant check alone does not close the window. Acceptance now requires that nothing is in flight anywhere — no pending REG2, no outstanding REG3 grant on any index — and reads the uplink's own `connected` flag, threaded through `process_registration_packet` from `packet_io.rs`. The probing branch is unchanged. Also records the round-3 certified HEAD (`ded2854`) in the task-15 evidence ledger, and appends the round-4 re-certification.
…rand the sender `handle_reg2` clears `pending_reg2_idx` and re-points `pending_timeout_at_ms` at a `REG3_TIMEOUT` deadline in the same statement block, but `clear_pending_if_timed_out` only fires `if let Some(idx) = self.pending_reg2_idx` — already `None` by then. The REG3-wait deadline was therefore set and never consulted, so an `awaiting_reg3` grant never expired. Round 4 made that permanent: `handle_reg_ngp` refuses to restart registration while any grant is outstanding, so a receiver that restarts between our REG2 and its REG3 answers the group it no longer knows with an entirely legitimate fresh REG_NGP, which the sender then refuses for the life of the process. Add `clear_awaiting_reg3_if_timed_out`, called from the same housekeeping tick right after `clear_pending_if_timed_out`: at the deadline it revokes every outstanding grant, zeroes the deadline, drops a queued REG2 rebroadcast (it would re-arm the grants just revoked), and re-opens the REG1 path. A pending REG2 cedes ownership back to the REG2 seam. The recovery is timeout-bounded, not an open door — the fresh REG1/REG2/REG3 cycle re-arms and re-gates. Tests: `expired_reg3_grant_lets_a_fresh_reg_ngp_restart_registration` drives the whole chain through `SrtlaConnection::process_packet`, including a second full cycle proving the gate re-closes; `housekeeping_expires_a_stale_reg3_grant` pins the production call site; `reg3_timeout_fires_at_4s_logical` no longer hand-restores `pending_reg2_idx` (it was silently exercising the REG2 seam) and now runs on the state a real REG1 → REG2 → REG2-broadcast leaves behind.
…annot strand the sender" This reverts commit 1680670.
…onnection state" This reverts commit ed7e74f.
…lback Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
…a port-0 peer Both send-failure helpers simulated a failing uplink by naming a port-0 peer. That is only synchronously invalid on Linux; macOS accepts an unconnected sendto to port 0, so the ten tests that exercise the send-failure recovery paths observed a successful send and never reached the code under test. The dependency is new to this branch: uplink sockets used to be connect(2)-ed, and connect() to port 0 is rejected uniformly across platforms. Unconnected sockets moved the validation to per-datagram sendto, where the semantics diverge. BatchUdpSocket now carries a cfg(test)/test-internals AtomicBool that makes every send path return a synthetic ConnectionRefused, so the trigger no longer depends on OS destination validation at all. Outside test builds the field is absent and the check is an inline(always) None, matching the ab_metrics zero-cost pattern. No assertion was weakened. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
irlserver/srtla_sendthroughc9f6bb2; branch is 0 behindmain.now_ms, RTT-velocity gate,sendmmsg/unconnected sockets, ACK-RTT ownership, EDPF velocity+BDP, DNS drift detection, and CLI-on-library deduplication.docs/notes/upstream-sync-2026-08-evaluation.md.Evaluation and registration hardening
AGENTS.mdunder ROBUSTNESS FIXES.Validation
The full authoritative gate is green per
.omo/evidence/task-15-upstream-sync-irlserver.md: build, fmt, clippy, tests, Loom, Miri, audit, deny, contract scripts, and the TypeScript binding gate.Merge policy
This is an upstream-sync PR. Per repository policy, upstream-sync PRs are never squashed: the true-merge history is the point of the change. Please merge with the true merge commit after CI passes; do not squash.