feat(rpc): add GET /lean/v0/events SSE stream - #517
Conversation
🤖 Kimi Code ReviewThe PR implements an SSE endpoint for chain events with proper best-effort semantics and clean separation between the blockchain actor (publisher) and RPC (subscriber). No critical issues found. Minor observations:
Security/Performance notes:
Architecture verification:
The implementation follows Rust async best practices and integrates cleanly with the existing axum router structure. Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt |
🤖 Codex Code Review
Otherwise the wiring looks clean, and the flat JSON framing test is a good regression check for the earlier double-tagging bug. I couldn’t run Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Claude Code ReviewReview:
|
Greptile SummaryThis PR adds a live SSE stream for blockchain events. The main changes are:
Confidence Score: 5/5This looks safe to merge.
|
| Filename | Overview |
|---|---|
| crates/net/rpc/src/events.rs | Adds the SSE handler, event framing, keep-alives, and payload test. |
| crates/net/rpc/src/lib.rs | Registers the events route and provides the shared event bus. |
| bin/ethlambda/src/main.rs | Connects one event bus to the blockchain publisher and RPC server. |
| crates/blockchain/src/events.rs | Adds direct serialization of chain events as flat JSON objects. |
Reviews (4): Last reviewed commit: "Merge branch 'main' into feat/events-sse..." | Re-trigger Greptile
🤖 Kimi Code ReviewOverall Assessment: Clean, idiomatic implementation of SSE streaming for consensus events. Correctly handles backpressure via bounded broadcast channel semantics and implements best-effort delivery as specified. Minor Issues & Suggestions: 1. Rust Version Compatibility (Line 47,
// Alternative for older Rust versions:
.json_data(&ev)
.map_err(|err| tracing::warn!(%err, "Failed to serialize SSE chain event"))
.ok()?2. Test Race Condition Risk (Lines 67-97,
3. Connection Resource Limits
4. Serialization Failure Handling (Line 47)
5. Documentation Accuracy (Line 27,
Security & Consensus Considerations:
Performance:
Acknowledgments:
Verdict: LGTM. Address the Rust version check for Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt |
🤖 Codex Code Review
I didn’t find other correctness issues in the diff. This PR doesn’t alter fork choice, attestation validation, finalization, state transition, XMSS, or SSZ logic; it only exposes existing I couldn’t run Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Claude Code ReviewConsistent with existing conventions. I have enough context to write the review now. Review:
|
23825f1 to
3a3c804
Compare
bb2acdd to
8bc49c0
Compare
🤖 Codex Code ReviewNo findings. The diff stays out of consensus-critical execution paths: it only wires the existing Residual risk: I couldn’t run Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Kimi Code ReviewThe PR adds a clean, well-tested SSE endpoint for chain events. Implementation follows Axum best practices and correctly handles slow clients via lag detection. Minor observations:
Security & Performance:
Consensus considerations:
Verdict: LGTM. The lag-handling logic is correct for a best-effort consensus event stream. Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt |
🤖 Claude Code ReviewReview:
|
3a3c804 to
a966632
Compare
8bc49c0 to
50050d2
Compare
First PR of the chain-events series (re-cut of #460), shipping only the pub-sub mechanism so the real design decisions (event shape, publisher model, back-pressure policy) are reviewable on their own. The SSE transport and topic filtering follow in separate PRs; the bus stays unconsumed until the endpoint lands. Emission is actor-layer only, not threaded through the store. Before each state-changing store call (store::on_tick, store::on_block) the actor snapshots (head, latest_justified, latest_finalized), runs the store function completely unchanged, then diffs the snapshot against the post-call state and emits in order block -> justified_checkpoint -> head -> finalized_checkpoint. This keeps store.rs, both spec-test files, and test_driver.rs byte-identical to main: no Option<&EventBus> parameter spreads across on_tick/on_block/update_head or any of their test call sites, and the sole-publisher property is structural (only BlockChainServer ever holds a live EventBus) rather than something every call site has to preserve by convention. Two consequences of diffing at this level, both accepted: - `block` needs a was-it-new guard: on_block returns Ok early for an already-imported root, so the actor checks block-known-ness (the same has_state check the store itself uses) before importing, and only emits `block` for a genuinely new root. - Several head moves inside a single tick or block import coalesce into one `head` event carrying the net move; fine for subscribers, since the beacon-API analog is also last-value-wins per slot. The bus itself is a facade over a single bounded tokio broadcast channel: emit() never blocks or fails, since a slow subscriber lags and drops instead of back-pressuring consensus, and a receiver-count guard makes emission free when nobody is listening. ChainEvent serializes untagged so payloads stay flat: the topic name travels out-of-band via ChainEvent::topic(), fixing #460's double-tagged SSE payload before any transport exists. The `head` event is additionally gated by HEAD_EVENT_RECENCY_SLOTS, adopted from a review of Lighthouse's canonical-head recompute path: a new head is only emitted when its slot is within that many slots of the wall clock, so startup catch-up and backfill don't spam subscribers with historical head moves on the way to the tip. `block`, justified_checkpoint, and finalized_checkpoint stay ungated; consumers tracking sync progress should watch `block` instead.
Second PR of the chain-events series: the transport for the event bus introduced in the previous PR. Kept separate from the mechanism so the axum plumbing reviews on its own; topic filtering (?topics=) follows in the next PR, so this stream is unfiltered. Each connection subscribes its own receiver and forwards events as SSE frames: the topic name goes on the event: line via ChainEvent::topic() (no hardcoded name match to drift from the enum) and the data: line carries the flat JSON payload, so the topic is never duplicated inside the body. A lagged client skips past the events the bounded channel overwrote (debug log, stream continues) rather than ending the stream: the stream is best-effort by contract and clients re-sync via the blocks endpoints. Keep-alive comments hold idle connections open through proxies.
50050d2 to
03118fc
Compare
a966632 to
60d9c2e
Compare
…nd_emit Reorder chain-event emission so the head event fires before justified_checkpoint, giving subscribers the head move ahead of the checkpoint updates that follow it. Drop the ordering assertion test, which only pinned the old sequence.
Colocate the pre-call snapshot type (and HEAD_EVENT_RECENCY_SLOTS) with the ChainEvent/EventBus definitions it depends on, keeping all chain-event machinery in events.rs. The actor references it via crate::events; the snapshot's diff/gate tests move alongside it. No behavior change.
The recency gate is already covered by chain_event_diff_gates_stale_head (gated) and chain_event_diff_emits_recent_head (within window); the explicit boundary case was redundant.
Eventless call paths (spec tests, test_driver.rs) never construct an EventBus at all, so the dormant-bus constructor had no production callers; its only user was its own test.
Shape ChainEvent payloads after the Ethereum beacon-API eventstream (v4.0.0): `block` is the block root, `state` the state root, and `slot` stands in for the beacon `epoch`. Drop the `#[serde(untagged)]` Serialize derive; the wire encoding (decimal-string ints, 0x-hex roots, out-of-band topic) returns with the SSE endpoint in a follow-up PR. This also removes the ambiguous untagged shape flagged in review. Since serde is no longer used in the crate, drop the serde/serde_json deps. `justified_checkpoint` has no beacon analog and is kept as an ethlambda extension mirroring `finalized_checkpoint`. Also fix a checkpoint event dropped on the proposer's own slots: the interval-0 catch-up in `get_proposal_head` advances head/finalized outside every snapshot window, so a finalization moved there was folded into the later block-import diff's baseline and never emitted. `propose_block` now snapshots around the build and emits those moves on both build success and failure, matching what a non-proposing node emits at its interval-0 tick.
#516 reshaped ChainEvent (root/parent_root -> block/state) and dropped its Serialize derive; move the wire encoding into an RPC-crate DTO.
#516 deferred the chain-event wire encoding to this SSE PR. Derive it on ChainEvent directly (untagged, flat body) rather than mirroring the enum in an RPC-crate DTO, so the handler stays a direct json_data(&ev) with no parallel type to keep in sync. Wire shape is unchanged.
Adopt #517's beacon-aligned ChainEvent ({slot, block, state}, Serialize derived on the type) while keeping #518's TopicSet/EventSubscription filtering. The snapshot tests moved from lib.rs into events.rs upstream; take that version and add TopicSet::ALL to their subscribe calls. Update the topic-filter tests to the new field names, and switch the "block filtered out" assertion to check the SSE event: line since every payload now carries a `block` field.
# Conflicts: # Cargo.lock # bin/ethlambda/src/main.rs # crates/blockchain/src/events.rs
🤖 Kimi Code ReviewOverall Assessment: Clean, well-documented implementation of the SSE endpoint. The use of Minor Issues & Suggestions: 1. Silent event dropping on serialization failure 2. H256 serialization assumption 3. Untagged enum ambiguity (acceptable but note) 4. Channel capacity visibility 5. Clone semantics of EventBus 6. Test fragility Security/DoS: No rate limiting on the SSE endpoint is visible. Consider adding a connection limit or maximum subscriber count to the Correctness: The lagged-client handling (skipping gaps) correctly implements the documented best-effort contract. The Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt |
🤖 Codex Code ReviewFindings
No fork-choice, attestation, justification/finalization, STF, XMSS, or SSZ logic changed in this diff, so I did not find a consensus-path regression beyond the new RPC/event-surface concerns above. I couldn’t run Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Claude Code ReviewReview:
|
Adds Beacon-API-style server-side filtering to `GET /lean/v0/events`: `?topics=` takes a **required** comma-separated list of topic names. As in the Beacon API `eventstream` endpoint, `topics` is mandatory: a missing, empty, or unknown value returns 400 (the body names the offending topic). There is no "subscribe to everything" default. See `docs/rpc.md`. Filtering lives in the SSE handler, the sole consumer that needs it, rather than in the bus. `EventBus::subscribe()` stays a plain fan-out (`broadcast::Receiver`); the handler parses the requested topics into a `Vec<Topic>` and skips unmatched events in its own `recv` loop. Lag is surfaced to the handler, never swallowed. Topic-name parsing (`FromStr for Topic`, `UnknownTopic`) stays in the blockchain crate, where the orphan rule requires it. The handler bridges `recv` to a stream via `futures_util::stream::unfold`, replacing the `tokio-stream`/`futures-core` pair with `futures-util` (already in the lock transitively). On `Lagged(n)` the stream also yields an `: error - dropped n messages` SSE comment frame (wire-compatible with Lighthouse's lagged-client marker) instead of only logging, so clients get an explicit re-sync trigger. Stacked on #517. Has unit and handler tests (including the missing/empty and unknown-topic 400 paths) and passes clippy with `-D warnings`.
Continues the chain-event pub-sub series (#516 / #517 / #518, all merged). Adds the beacon `block_gossip` analog. ## Event added | Topic | Payload | Emitted when | |-------|---------|--------------| | `block_gossip` | `{ slot, block }` | A block is seen on the network, before import | ## Notes - Emitted from the blockchain actor's `NewBlock` handler, **not** a second P2P-side publisher, so the actor stays the sole event publisher and the write flow stays one-directional (a core requirement of the series' design). - Fires for both gossip'd and by-root-fetched blocks, since both re-enter through `NewBlock`. Import may pend the block while its parent chain is fetched, so this precedes import. - Ungated like `block` (not recency-gated), so subscribers can watch sync progress. Low-rate (one per block), so the payload is built unconditionally behind `emit`'s no-subscriber guard. ## Testing `cargo fmt`, `cargo clippy -p ethlambda-blockchain -p ethlambda-rpc` (clean), blockchain lib + rpc events tests pass. --- **Independent PR, based off `main`.** One of three sibling PRs continuing the series — alongside #533 (`chain_reorg`/`safe_target`) and #534 (`attestation`/`aggregate`), each independently based off `main`. They touch overlapping regions of `events.rs` / `docs/rpc.md`, so whichever two merge later will each need a small conflict rebase. Opened as draft.
Continues the chain-event pub-sub series (#516 / #517 / #518, all merged). Adds the two high-rate topics. ## Events added | Topic | Payload | Emitted when | |-------|---------|--------------| | `attestation` | `{ validator_id, data: AttestationData }` | A single validator vote passes gossip validation (signature omitted) | | `aggregate` | `{ participants: [u64], data: AttestationData }` | A committee-signature aggregate is produced locally or accepted from gossip (proof omitted) | Both fire only for messages the store **accepted** (data + signature validation), so subscribers see the same votes fork choice does. A node never receives its own aggregate back over gossip, so `aggregate` fires at two sites: the locally produced `AggregateProduced` path and the gossip-received path. ## Design points - The ~3 KB XMSS signature and the SNARK proof bytes are deliberately **omitted** — too heavy for a high-rate stream. - The shared broadcast channel capacity is bumped **256 → 8192**. All topics share one ring buffer, so a subscriber's tolerable stall is `capacity / total_event_rate` (dominated by the attestation rate), not per-topic. A per-topic split behind the `EventBus` facade remains the escape hatch if real devnet rates show the shared window biting — see the note in `docs/rpc.md`. - Emission uses the plain, no-subscriber-guarded `emit`. Consequence: the per-vote `AttestationData` is cloned even when nobody is subscribed (the guard only skips the send). If that actor-hot-path cost proves material, a payload-deferral helper can be added in a follow-up. ## Testing `cargo fmt`, `cargo clippy -p ethlambda-blockchain -p ethlambda-rpc` (clean), blockchain + rpc lib tests pass, including `events_streams_attestation_with_nested_data` (verifies the untagged `data:` nesting over the wire). --- **Independent PR, based off `main`.** One of three sibling PRs continuing the series — alongside #533 (`chain_reorg`/`safe_target`) and #535 (`block_gossip`), each independently based off `main`. They touch overlapping regions of `events.rs` / `docs/rpc.md`, so whichever two merge later will each need a small conflict rebase.
## Summary Adds `tooling/event-monitor/`, a standalone live arrival-time dashboard for lean-consensus (ethlambda) nodes. It dials the `GET /lean/v0/events` SSE stream of several nodes, timestamps each event on a single collector clock, and serves a browser dashboard visualizing: - a **rolling beeswarm** of each event's arrival offset *within the slot*, one lane per node, and - a **propagation-delta beeswarm**: for a given block / aggregate, how long after the *first* node each other node saw it. The rolling window is adjustable live from the header, and a fresh page load backfills recent history from the collector so it is never blank. ## Design - **Standalone Cargo workspace** under `tooling/event-monitor/` (empty `[workspace]` table); it is not a member of the parent `ethlambda` workspace and does not affect the main build. - **Zero dependency on any ethlambda crate** — it only speaks the documented SSE/HTTP wire shape. `CONTRACT.md` is the frozen interface between the Rust/axum backend (`src/`) and the vanilla-JS frontend (`web/`, no build step). - A single collector clock makes propagation deltas skew-free across nodes. - `?demo=1` runs the frontend fully offline with synthetic data. This PR contains the tooling plus the CI wiring needed to actually check it. The RPC / event-bus changes that make a node emit these events shipped separately in the chain-events series (below). ## Required ethlambda API functionality All dependencies are **satisfied on `main`** — the default dashboard view works against a current node with no extra PRs. | Capability | Endpoint / topic(s) | Status | |---|---|---| | SSE events stream | `GET /lean/v0/events` | ✅ #517 | | Topic filtering | `?topics=<csv>` | ✅ #518 | | Slot-geometry bootstrap | `GET /lean/v0/genesis`, `GET /lean/v0/config/spec` | ✅ | | Base topics | `block`, `head`, `justified_checkpoint`, `finalized_checkpoint` | ✅ #516 | | Attestation / aggregate topics | `attestation`, `aggregate` | ✅ #534 | | Block-gossip topic | `block_gossip` | ✅ #535 | The collector accepts `block`, `block_gossip`, `head`, `justified_checkpoint`, `finalized_checkpoint`, `attestation` and `aggregate`; any other topic is logged and skipped. `safe_target` / `chain_reorg` (#533, closed) were removed in 4eeafff — both panels' topic filters discarded them, and `chain_reorg` is a point-in-time event rather than a per-node arrival race, so it does not belong on a beeswarm. Both panels default to `block` / `attestation` / `aggregate`. ## Review fixes Three defects found in review, each with a regression test: - **Reconnect backoff never reset after a healthy session.** The reset was keyed on a clean end-of-stream, but a node restart surfaces as a stream *error*, so every restart ratcheted the delay one step permanently; after ~7 of them a healthy node reported `down` with 10s reconnects. Now keyed on session duration (survived ≥1 heartbeat). This also closes the inverse case: a peer that accepts the request and instantly closes the stream used to be retried at `INITIAL_BACKOFF` forever. - **One bogus slot blanked the dashboard.** Both the history ring and the rolling window key retention off the highest slot seen, and that watermark only moves up, so a single event from a node on a different genesis aged out every real event until a restart. Events more than `MAX_FUTURE_SLOTS` ahead of the collector's own slot are now dropped, warning once per node per connection. Only the future side is bounded; old slots are legitimate (`finalized_checkpoint` trails head) and cannot move the watermark. - **Stale status clobbered fresh status.** The frontend applies `status` immediately but buffers `chain` during backfill, so the `/api/history` snapshot could overwrite a newer live status. Live status now wins. Also from that review: - **Slot geometry is re-resolved every 60s** and republished when it changes, dropping the retained history and slot watermark, so a regenerated genesis no longer silently corrupts every subsequent `offset_ms`. Collectors re-read per frame and `/api/meta` per request; an already-open tab needs a reload to pick up new geometry. - **The arrival axis now spans two slots at two scales**: the first at full resolution (90.9% of the width at 5 intervals), the next compressed into a tinted band half a first-slot interval wide, saturating beyond that. A block spilling past its slot boundary is the failure mode worth seeing, and it used to be indistinguishable from one landing exactly on the boundary. - **`offset_ms` includes the collector↔node round trip.** One clock is what makes propagation deltas skew-free, but nodes reached over different links carry a systematic offset that reads as lag. Now documented in `README.md` and `CONTRACT.md §2`. - History events are `Arc`-shared, so `publish_chain` no longer deep-copies per event and `/api/history` no longer clones up to `HISTORY_MAX_EVENTS` events while holding the mutex. - `topics = []` / `nodes = []` are rejected at load rather than becoming an opaque retry loop against a 400. Known and deliberate, left as follow-ups: the collector cannot see upstream `: error - dropped N messages` comments (`eventsource-stream` swallows them), so a lagging subscription silently under-samples; both canvases still repaint unconditionally at 60fps; and `bootstrap` has no retry, so the monitor exits if started before its nodes are listening. ## CI `tooling/event-monitor` declares its own `[workspace]`, so `cargo fmt --all`, `cargo check --workspace`, `cargo clippy --workspace`, `make lint` and `make test` all stop at the root workspace members and never reached it — it landed with nothing verifying it. The `Lint` job now also runs `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and `cargo test` under `working-directory: tooling/event-monitor`, and `rust-cache` lists it as a second workspace so its target dir is cached and its `Cargo.lock` feeds the cache key. ## Testing - `cargo test` in `tooling/event-monitor/`: **41 passed** (39 lib + 2 integration), 0 failed. - `cargo fmt --check` and `cargo clippy --all-targets -- -D warnings`: clean. - `?demo=1` offline synthetic mode renders both panels with no live nodes. - Verified end-to-end against a live local devnet (blocks land ~0.1–0.2 s into the slot, attestations ~0.8 s = interval 1). - Two-slot axis geometry checked numerically: the overflow band is exactly 0.5 × one first-slot interval, the mapping is monotonic, and the endpoints land on the margins. - Geometry refresh checked against a stub node: rewriting its `genesis_time` mid-run is picked up within one refresh interval (`WARN slot geometry changed`), `/api/meta` then serves the new epoch, and the history ring holds the restarted chain's low slots with in-slot offsets — which only works because the watermark is reset, otherwise they would prune as older than the previous high-water mark. ## Run ```bash cd tooling/event-monitor cp config.example.toml config.toml $EDITOR config.toml # list your nodes' RPC URLs + topics cargo run --release -- --config config.toml # open the `listen` address (default http://127.0.0.1:8080) ```
Adds
GET /lean/v0/events, a Server-Sent Events stream of the chain events introduced in #516:head,block,justified_checkpoint,finalized_checkpoint. Every subscriber currently receives all events; server-side?topics=filtering lands in the next PR of the series.Frames carry the topic on the SSE
event:line and a flat JSON payload indata:(built fromChainEvent::topic(), no hardcoded name mapping). A client that reads too slowly skips the events it missed (debug-logged, best-effort; re-sync via the blocks endpoints) rather than back-pressuring the actor. Keep-alive comments hold idle connections open. Documented indocs/rpc.md.Stacked on #516.
Has handler tests and passes clippy with
-D warnings.