Skip to content

feat(ws): add fail-closed ordered market stream and shutdown - #110

Open
yovanoc wants to merge 8 commits into
Polymarket:mainfrom
yovanoc:fix/public-market-stream-shutdown
Open

feat(ws): add fail-closed ordered market stream and shutdown#110
yovanoc wants to merge 8 commits into
Polymarket:mainfrom
yovanoc:fix/public-market-stream-shutdown

Conversation

@yovanoc

@yovanoc yovanoc commented Aug 28, 2026

Copy link
Copy Markdown

Closes #109.

Motivation

Consumers maintaining a local executable orderbook need the initial BookUpdate, every batched PriceChange, and related market events in one connection-ordered stream. Merging the existing typed streams can miss/reorder the initial snapshot and deltas; lag and malformed interested frames previously disappeared without a terminal error. Downstream runtimes also had no way to await WebSocket/reconnection shutdown.

Changes

  • adds Client::subscribe_market_events and _with_options, returning one ordered Stream<Item = Result<WsMessage>>;
  • registers each internal receiver before sending the subscription request;
  • serializes subscribe/unsubscribe/reconnect state and rolls refcounts/interest/auth/custom-feature state back on setup failure;
  • gives active subscriptions unique identities;
  • surfaces broadcast lag as terminal WsError::Lagged;
  • carries valid messages and parser errors through one ordered internal ConnectionEvent stream while preserving the existing public ConnectionManager::subscribe() -> Receiver<M> API;
  • stops silently dropping malformed interested batch entries; unknown enum values such as Side::Unknown remain visible to callers;
  • uses a weak reconnect-handler reference to remove the retention cycle;
  • adds idempotent, concurrent-safe Client::shutdown() / close() that blocks new channel creation during shutdown, stops and awaits reconnect/connection tasks, and permits clean later reuse;
  • makes socket writes cancellable by requested shutdown;
  • clears stale user auth/interest after final unsubscribe;
  • applies the same fail-closed lag/parser behavior to RTDS subscriptions.

Unsubscribe remains transport/refcount management; callers should drop a returned stream when they no longer want to consume it. Channels stay alive to flush unsubscribe messages and support reuse until explicit shutdown or final Client drop.

Tests

  • ordered Book -> PriceChange delivery on the unified stream;
  • unknown market side is surfaced, not dropped;
  • malformed interested market frame terminates with an error;
  • concurrent/idempotent shutdown, disconnected state, and post-shutdown reuse;
  • all existing reconnect, multiplex, unsubscribe, market, user, and RTDS tests.

Validation:

  • cargo test --all-features --no-fail-fast — 611 passed, 20 ignored
  • cargo test --all-features --test websocket — 45 passed
  • cargo clippy --all-features --lib -- -D warnings
  • cargo clippy --all-features --test websocket -- -D warnings
  • cargo check --all-features --workspace --all-targets (two pre-existing unused imports in unrelated order tests)

Note

Medium Risk
Touches core WebSocket subscription, reconnect, and shutdown paths; behavior changes (lag/parse fail-closed, channel retention, all-markets user semantics) can affect long-running orderbook consumers.

Overview
Adds subscribe_market_events (and _with_options) so book snapshots, price deltas, and related market frames share one connection-ordered Stream<Item = Result<WsMessage>>, plus unsubscribe_market_events and Client::isolated() for independent channel lifecycles.

Lifecycle and transport: shutdown / shutdown_if_idle / close are idempotent and coordinate with a lifecycle lock so idle shutdown cannot race subscription setup; channels are no longer torn down automatically when refcounts hit zero (unsubscribe flush and reuse until explicit shutdown). ConnectionManager gains cancellable shutdown, a larger broadcast buffer, and an internal ConnectionEvent path (messages, parse errors, shutdown) while keeping the public subscribe() API.

Fail-closed streaming: Subscriptions register receivers before wire requests; setup failures roll back refcounts/interest/auth. Broadcast lag and parse errors end the stream with WsError::Lagged / InvalidMessage instead of continuing silently. Interested batch parse failures in parse_if_interested propagate rather than being skipped. RTDS picks up the same event stream behavior and ignores control acks without topic/type/payload.

User channel: Empty markets (“all markets”) subscriptions are refcounted separately so targeted unsubscribes and reconnect resubscribe do not break the global stream.

Reviewed by Cursor Bugbot for commit 39a3fe3. Bugbot is set up for automated code reviews on this repo. Configure here.

@yovanoc

yovanoc commented Aug 28, 2026

Copy link
Copy Markdown
Author

Downstream PMKit dogfood found and exercised three lifecycle needs now included in follow-up commits:

  • c20f562: matching unsubscribe_market_events for the unified stream;
  • 217704f: atomic shutdown_if_idle, so one adapter cannot terminate active sibling streams sharing the same Client;
  • 8f39ed1: local refcount/interest/auth cleanup completes even when the unsubscribe frame cannot be sent.

The complete PMKit runtime integration now reconstructs token-scoped snapshot+deltas from this stream and passes 438 workspace tests, including GTD execution regressions against SDK 0.7. SDK validation remains 163 library + 47 WebSocket tests and strict lib clippy.

@yovanoc

yovanoc commented Aug 29, 2026

Copy link
Copy Markdown
Author

Full PMKit dogfood completed using #110 plus #105 on fork integration branch integration/rtds-twap-market-stream. Real-traffic follow-ups on #110 now include topicless RTDS control handling, isolated public clients for per-token initial snapshots, and bounded burst capacity. Combined SDK tests: 169 lib + 48 WebSocket. A complete 15-minute PMKit window processed 261,088 market events with zero source gaps; exact RTDS E18 evidence was preserved. No orders were placed.

@yovanoc
yovanoc marked this pull request as ready for review August 29, 2026 13:01

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

Reviewed by Cursor Bugbot for commit 109abdd. Configure here.

Comment thread src/clob/ws/subscription.rs
Comment thread src/clob/ws/client.rs
@yovanoc

yovanoc commented Aug 29, 2026

Copy link
Copy Markdown
Author

Addressed both Cursor findings in 39a3fe3 and adversarially checked the follow-up. Validation: cargo test --all-features --no-fail-fast (621 passed, 20 ignored), strict all-feature library clippy, fmt, and clean LSP for both edited library files. The large integration test file is covered by Cargo but timed out in rust-analyzer. Ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose one fail-closed public market event stream for local orderbook reconstruction

1 participant