fix(tn-reth): derive per-worker RPC/IPC endpoints in start_rpc (#1287) - #1298
Open
MavenRain wants to merge 8 commits into
Open
fix(tn-reth): derive per-worker RPC/IPC endpoints in start_rpc (#1287)#1298MavenRain wants to merge 8 commits into
MavenRain wants to merge 8 commits into
Conversation
Every worker started the one shared NodeConfig.rpc: the second worker
silently unlinked worker 0's IPC socket (reth removes the endpoint path
before it binds) and a fixed http/ws port failed startup with AddrInUse.
Derive each worker's RpcServerArgs instead: worker 0 keeps the operator's
values; worker w shifts an enabled http port down by 200*w and an enabled
ws port up by 400*w (the stride clears reth's whole --instance range) and
suffixes an enabled IPC path with -w{w}. Port 0 stays OS-assigned, a
shift that leaves the valid range fails startup loudly, and the resolved
endpoints are logged per worker at info.
Cross-transport distinctness is enforced up front: fixed bases need
ws_port >= http_port (equality is reth's shared http+ws server), since
inverted bases would let one worker's http band land on another worker's
ws band, and a derived http port stops above the privileged range so the
failure names the worker and offset instead of a bare EACCES at bind.
Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
…-network into tn-1287-per-worker-rpc-endpoints
MavenRain
requested a deployment
to
merge-into-main
August 27, 2026 17:46 — with
GitHub Actions
Waiting
MavenRain
requested a deployment
to
merge-into-main
August 27, 2026 17:46 — with
GitHub Actions
Waiting
…c-endpoints Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
23 tasks
…c-endpoints Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
…c-endpoints Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
…c-endpoints Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
…c-endpoints Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
…c-endpoints Round 6: main 66e0d14, 8 commits past the round-5 target 87364b2 (#1270, #1295, #1291, #1288, #1280, #1268, #1294 and one fork merge; 38 files). Two files overlap the PR. engine/inner.rs auto-merged: main's WorkerBaseFee handle at get_rpc_server, the PR's worker_id at start_rpc. env/rpc.rs needed two hand reconciliations because #1295 changed get_rpc_server to take a WorkerBaseFee handle instead of a BaseFeeContainer clone: - The tn_types import: keep the PR's WorkerId and take main's WorkerBaseFee. - The PR's start_worker_rpc test helper: take a &GasAccumulator and pass worker_id's container to init_txn_pool and its worker_base_fee handle to get_rpc_server. The two-worker IPC test builds one GasAccumulator::new(2) so each worker resolves its own slot, matching main's fee_history_methods_for_worker helper. Every other file is byte-identical to origin/main. Signed-off-by: Onyeka Obi <softwareengineerasaservant@isurvivable.cv>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1287. (Cantina #23)
Problem
RethEnv::start_rpcread the one process-wideNodeConfig.rpcon every call, andExecutionNodeInner::initialize_worker_componentscalls it once per worker. With more than one worker (the #554 to #559 track) every worker's RPC server targeted the same http/ws socket and the same IPC path:debug!.AddrInUse, whichinitialize_worker_componentspropagates as a fatal startup error, after the IPC steal already happened.--with-unused-portsdid not close the gap: it randomizes the IPC path once per process, so every worker still shared it. No attacker is required for either failure; the default configuration failed silently rather than loudly.Not a bug on
maintoday (onlyDEFAULT_WORKER_IDis ever initialized). This is the design gap #558 has to close, landed as its precursor.Fix
crates/tn-reth/src/env/rpc.rs:start_rpctakes theworker_idand derives that worker'sRpcServerArgsfrom the operator's config (worker_rpc_server_args) instead of reusing the shared one:w > 0shifts an enabled http port down by200 * w, shifts an enabled ws port up by400 * w, and suffixes an enabled IPC path with-w{w};--instancebound, not guessed: instance offsets span at most 199 ports (instance <= 200in reth v1.11.3), so a 200-port worker stride keeps every(instance, worker)pair on a distinct port, and the-w{w}IPC suffix is disjoint from instance's-{i}by construction;--with-unused-portssentinel) passes through, since the OS assigns a distinct port per bind, and the flag's one random IPC path still gets the per-worker suffix;TnRethError::WorkerRpcPort) instead of wrapping into a port another worker or instance owns; a derived http port additionally stops above the privileged range (below 1024 a non-root process cannot bind), so that failure also names the worker, base, and offset instead of dying later in the bind with a bare permission error;ws_port >= http_port(reth's default layout; equality is the shared http+ws server): http bands stride down and ws bands stride up, so inverted bases would let one worker's http band land on another worker's ws band; the first derived worker fails loudly (TnRethError::WorkerRpcPortOrder) and worker 0 still binds inverted bases as configured, keeping single-worker nodes working;crates/tn-reth/src/env/rpc.rs: the resolved endpoints (http/ws/ipc) are logged per worker atinfo!once the server starts.crates/node/src/engine/inner.rs:initialize_worker_componentspasses itsworker_idtostart_rpc.crates/tn-reth/src/error.rs: newWorkerRpcPortandWorkerRpcPortOrdervariants carrying the worker id (WorkerId), the transport, and the offending ports and offset.No wire change:
RpcNodeInfocarries no endpoints.Testing
test_worker_zero_keeps_operator_rpc_endpoints: worker 0's derived args equal the operator's.test_worker_endpoints_are_distinct_per_worker: workers 1 and 2 derive distinct http/ws ports and IPC paths with the exact band arithmetic.test_worker_bands_clear_the_instance_range: the closest approach between the worker and instance schemes (worker 1 of instance 1 against worker 0 of instance 200) stays separated on both transports.test_out_of_range_worker_port_is_a_loud_error: an enabled http port below the band and an enabled ws port atu16::MAXboth fail withWorkerRpcPort, not a wrapped port.test_zero_ports_stay_os_assigned_and_ipc_still_suffixes:--with-unused-portssemantics survive derivation.test_inverted_transport_bases_are_a_loud_error: fixed ws below fixed http fails the first derived worker withWorkerRpcPortOrder; worker 0 keeps the inverted config as given.test_equal_transport_bases_stay_valid: reth's shared http+ws server (equal ports) derives non-colliding split bands.test_http_band_below_the_privileged_floor_is_a_loud_error: a derived http port under 1024 fails at derivation time, not at bind time.test_disabled_transports_keep_their_configured_values: a disabled transport never shifts and never errors; disabled IPC keeps its path.test_two_workers_bind_distinct_live_ipc_sockets(unix): two workers started on oneRethEnvleave both IPC socket files on disk; before this change the second start unlinked worker 0's.Ran nightly
cargo fmt -- --check,cargo check --workspace --all-targets, andcargo nextest run -p tn-rethon the rpc tests locally. Each new load-bearing test was seen to fail with the derivation reverted to the shared config and pass with it restored.