Repository navigation
refactor(book-feeds)!: port the five venues onto the snapshot feed layer - #1561
die-herdplatte wants to merge 32 commits into
Conversation
The five venues Tycho prices off the chain — Bebop, Hashflow, Liquorice,
Native and Metric — were all filed under `rfq:`, a prefix that names a
contract only four of them have: Metric fills from its own pool and signs
nothing. `book:` states what all five share and what consumers actually
branch on, which is that pricing arrives as a complete book the venue pushes
over a client-side feed, where `vm:`, a bare name and `pricelevelstream:` do
not. It also matches the `book/` module and the `book-feeds` feature.
Renaming only Metric was the cheap alternative and does not work: `pamm:`
names the venue kind, which distinguishes nothing — `vm:bopamm`,
`propammfallback:fermiswap` and `pricelevelstream:{pamm}` are pAMMs too —
and any second name on this axis makes `rfq:` a specialization of `book:`
rather than a sibling, because the two have the same data source. One family
name, and the execution path stays out of the identity.
`pricelevelstream:` keeps its own prefix: its suffix is a third-party feed's
venue, not the party serving the book.
The cargo feature `rfq` folds into `book-feeds`, which already gates the
`book` layer and the transport loops, and gains the dependencies the
providers need. The in-repo names follow: the example is `book_quickstart`,
the integration test's processor is `BookStreamProcessor` over a
`UpdateType::Book`, and its credential enum is `RfqVenue` with the four
venues that sign — Metric never entered it, and the fifth variant existed to
be excluded. The flags become `--disable-rfq-feeds` and
`--disable-pamm-feeds`: two subsets of the book family, replacing
`--disable-rfq` beside `--run-pamm-protocols`, which was a
`default_value_t = true` bool that clap could never set to false, so Metric
could not be turned off at all.
No alias for the old spelling: a deployment passing `--disable-rfq` or
configuring `rfq:bebop` fails loudly instead of silently changing meaning.
`substreams/executors.sql` inserts with `ON CONFLICT DO NOTHING`, so an
existing analytics database needs a DELETE of the five old rows and a
re-apply.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bebop, Hashflow, Liquorice, Native and Metric publish their complete current book over a tokio::sync::watch channel, from a feed task that owns the connection and is polled whatever the consumer does. SnapshotFeed is the whole streaming surface: a feed publishes through a Publisher only the library can make, so the channel and the task belong to whoever reads the feed. It replaces a path that served no one. Each client serialised its book into indexer wire-format snapshots, TychoStreamDecoder decoded them back in the same process, and RFQStreamBuilder merged the results through a bounded queue — so the socket was polled only when the consumer polled, and a consumer that fell behind by more than Bebop's pong deadline lost the connection. The same round trip reported every component as new on every frame: ~576 phantom additions announced to everything downstream, 32 times a minute, indefinitely. Shared behaviour lives once in snapshot_feed/ and book/: the WebSocket and HTTP feed loops with their failure counting, backoff and stale withdrawal, BookFeedConfig and the two transport configs, price-level validation, swap math and component ids. Each provider is reduced to its feed builder, its book source and its API client, and the five per-protocol decoders are gone. Metric moves to pamm/, which is where a venue priced off the chain but executed against its own pool belongs: the line to rfq/ is the counterparty, not the transport. Measured against 0.387.0, both driving Fynd on the live Bebop book: 23-39% less CPU for the same work across four configurations, at identical throughput and cadence. BREAKING CHANGE: RFQClient, RFQStreamBuilder and the per-protocol decoders are removed. A feed is a *Feed built by *FeedBuilder::new(BookFeedConfig, ...) implementing SnapshotFeed, whose run(publisher) publishes BookSnapshot<ReceivedAt> through a Publisher it is handed and cannot construct, so reading a feed is the only way to run one: as a BookFeedStream, a labelled BookFeedStreams, or a BookFeedWatch handing out receivers. Every pair-keyed component id changes — it is the protocol system and the two addresses as raw bytes, where it used to be a keccak256 of a similar string. States are constructed by feeds only and their serialized forms change, credentials are constructor arguments because the library no longer reads the environment, and every venue validates its price levels at ingestion. The PR description lists every item. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`WsSource::request` returns a `FeedError`, which classifies itself through `is_fatal` — one kind another attempt may answer differently, one it will not. The loop ignored the distinction and ended the feed on either, where `decode` on the same trait and `fetch` on the polling side both route the retryable kind through the failure budget. Opening a connection is one `Result` now: the handshake the source builds, the connection attempt and its deadline all produce either a stream or the reason there is none, and that reason reaches the budget that was already there. Only `is_fatal` short-circuits. `record_failure`, the warning and the backoff stay in the one place they were. Both halves of that rule are pinned, by one source that refuses the handshake every time and two tests that differ only in how it classifies the refusal. `a_request_the_source_cannot_build_counts_against_the_budget` takes the retryable half through the message the feed gives up with, which names the streak: "gave up after 2 consecutive failures: request failed: …" rather than the bare error. `a_handshake_the_source_will_never_build_ends_the_feed_at_once` takes the other on an unlimited budget, where the classification is the only thing that can end the feed, and asserts the error arrives as the source wrote it. Restoring the old `?` fails the first; dropping the `is_fatal` arm fails the second. That second test is the one with a venue behind it: of the five #1561 ports, only Bebop reads this loop, and its `request` returns `Fatal` alone — so no venue changes behaviour here today, and the path that does run was the one nothing covered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
## [0.446.0](0.445.1...0.446.0) (2026-10-08) ### ⚠ BREAKING CHANGES * **snapshot-feed:** report a panicked feed task as a message ### Features * **snapshot-feed:** add the latest-value snapshot feed layer ([cf06a5a](cf06a5a)) * **snapshot-feed:** add the latest-value snapshot feed layer ([#1554](#1554)) ([5472845](5472845)) ### Bug Fixes * **snapshot-feed:** count a handshake the source could not build ([7dbf14c](7dbf14c)), closes [#1561](#1561) ### Code Refactoring * **snapshot-feed:** report a panicked feed task as a message ([8b14925](8b14925))
The protocol-system rename is a separate change with a separate audience.
Everything else in this PR breaks library consumers — signatures, state
serialization, component ids — which a compiler or a resync settles. Renaming
the five systems breaks deployments: a solver's `--protocols`, an allowlist, a
dashboard filter, a saved query, each hand-edited in step with the upgrade.
Bundling the two spends the expensive break early, and on one of several
prefixes that encode different things in the same string.
So the five constants are `rfq:bebop`, `rfq:hashflow`, `rfq:liquorice`,
`rfq:native` and `rfq:metric` again, wherever they are produced or consumed:
the provider module constants, tycho-execution's encoder registry, gas
estimator and five swap encoders, the five config JSONs,
`deploy-executors.js`, `executors.sql`, the integration tests and both gitbook
pages. The component-id tests used a synthetic `book:venue`; they use
`rfq:venue` now, with the expected bytes recomputed.
Component ids still change against main, which hashed `"<venue>_{a}/{b}"` with
keccak256 — they are now `rfq:bebop` followed by the two addresses as raw
bytes.
What does not depend on the prefix stays: the `book-feeds` feature absorbing
`rfq`, `--disable-rfq-feeds` and `--disable-pamm-feeds` in place of
`--run-pamm-protocols` (a `default_value_t = true` bool, so clap could never
set it false and Metric could not be switched off), `RfqVenue` with the four
variants that take credentials, and the `book_quickstart` and
`book_stream_processor` names, which follow the `book/` module and the
`book-feeds` feature already on main.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Brings the review round on the feed layer (#1554) and 85 commits of main onto the venue ports. Nine conflicts; six were two architectures meeting in one filename, three were the review landing on files this branch rewrote. The review's changes are taken whole: `Panicked(String)`, `book_label`, the trimmed `max_snapshot_age` doc, `tvl_in_usd`. Two things the review removed return here, which is where they were always going: `test_token_map`, called by eight provider files, and `test_support`'s per-response delay, request counter and target routing, which the venue tests drive mock servers with. The `expect(dead_code)` attributes that said no feed existed yet are gone with the feeds they were waiting for. `client_builder.rs` and Metric's old `client.rs` stay deleted, and the Bebop and Hashflow `client.rs` stay the quoting clients this branch made them — main edited the feed clients that used to live under those names. What main added there has to come across, and does: the venue fallback routers (#1518), live since 2026-10-08. `with_fallback_router()` is on the Bebop, Hashflow and Metric builders as before, so a caller needs no change, but the mechanism underneath follows this branch's shape. Main stored a `protocol_system` on the client and overwrote it; here the system is resolved once by the builder and carried on the source, which stamps every component with it — so the choice is made in one place per feed rather than mutated after construction. Each venue gains `FALLBACK_PROTOCOL_SYSTEM` beside its `PROTOCOL_SYSTEM`, and the assertion that pins it to tycho-execution's own copy of the literal comes across with it: the two crates each hold the string, and nothing else holds them to one spelling. Note that a fallback-routed component now also has a different id, since this branch derives the id from the protocol system. Main's ids follow the same field, so the relation is unchanged; the values differ as they do for every other component here.
A quote timeout under a second reported itself as "timed out after 0 seconds", because the message formatted `as_secs()`. `Duration`'s `Debug` writes the unit it needs — 500ms, 5s, 15s — so a sub-second deadline reads as what it is and a whole-second one is unchanged. Seven messages: the quote-request deadline on Bebop, Hashflow and Liquorice (two arms each, the pre-attempt check and the post-attempt one) and Metric's per-pool bid/ask deadline. Native already did this, and its `timed_out_after_50ms` assertion is what the other three would have needed to catch it. The two remaining `as_secs()` calls stay: Liquorice's quote expiry and Native's deadline check are unix timestamps in seconds, not durations being described to a reader. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five aliases fill the book types into the feed readers, and two of them had no user: consumers reached past `BookFeedOutcome` to `SnapshotFeedOutcome`, and nothing named `BookFeedStream` at all, while `BookFeedStreams` and `BookFeedEvent` were used throughout. A half-adopted set is worse than either whole: a reader of one feed had to write `SnapshotFeedStream<BookSnapshot<ReceivedAt>, FeedError>` where a reader of several wrote `BookFeedStreams`. So the users come to the names. The quickstart, the integration-test processor and the `BookFeedStreams` doctest match on `BookFeedOutcome`, which drops an import from each; the solver guide reads a single feed as a `BookFeedStream`, beside the `BookFeedStreams` and `BookFeedWatch` it already used in the same paragraph. The integrator guide keeps `SnapshotFeedStream`: it addresses a venue implementing the trait, who need not be in the book family at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three places describe the family by its prefix rather than by a venue, so
swapping `book:{venue}` for `rfq:{venue}` left them behind. tycho-execution's
registry note had become self-contradictory, labelling the family `book:`
beside its own `rfq:bebop` and `rfq:metric` examples; the integrator guide
promised components stamped `book:<venue>`; the solver guide pointed at a
table of `rfq:` systems and called them `book:`.
The library's own vocabulary is untouched and stays: the `book/` module, the
`book-feeds` feature and the `Book*` types describe what these venues have in
common, which is a complete book from the venue, whatever the components are
called on the wire.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four builders pointed at `HttpFeedConfig` / `WsFeedConfig` for "the shared values", but those types document the fields and hold no values: the literal lives in `default_http_feed_config()`, which is crate-private and absent from rustdoc. Each venue's doc now says which of the two cases it is in instead — the shared set because nothing about that venue has been measured, or its own and why, which is Metric. The spreading example was prose in all five places and is a doctest in all five now, so the builder path, the config type and the field name are checked rather than described. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One price level a venue got wrong cost a poll every pair in the same response, including pairs outside the configured token universe. Bebop sends its whole book in one frame every ~1.7 s, so a pair it kept mispricing stopped the feed from publishing at all: the frame was dropped, the connection recycled, and `max_snapshot_age` withdrew the book 30 s later. Entries are now read one at a time and the ones that fail are dropped with a warning naming how many went and why the first did. A response where entries were dropped and none survived stays a parsing failure: a feed publishes complete snapshots, so serving one with nothing in it would withdraw every pair the venue had, which a malformed response must not be able to do. `book::wire::EntryReader` holds that rule for all four venues. The three that send JSON reach it through `SkipInvalidEntries`, a `serde_with` adapter on the collection their entries arrive in; Bebop reaches it directly, since its frames are protobuf and its pairs are already decoded when `BebopBook::try_from` rejects one. Only Native rejected an invalid level before this branch, and it rejected the whole response too. Bebop and Hashflow accepted such levels and Liquorice silently dropped malformed pairs, so the all-or-nothing rule was itself new here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Hashflow and Liquorice took whatever chain they were configured with and put it straight into their client. With the default `max_consecutive_failures: None`, a chain the venue does not price became an endless failed poll rather than a build error, while the PR names an unsupported chain as *the* example of a fatal one. Worse, both reached the chain through `Chain::id()`, which panics for a custom chain missing from the registry, on every poll and every quote — so the failure was a panicking feed task, not only a retry loop. Each builder now resolves the chain once, and which half does the deciding follows from how the venue's API is addressed. Bebop and Native carry the chain by *name* in their endpoints, which no chain id can supply, so the list stays with us: `<Venue>SupportedChain` in the venue's `models.rs`, and `build()` fails `FeedError::Fatal` for anything else. Bebop's list is the eight chains its API prices rather than the two Tycho can execute on — whether a swap can be encoded is tycho-execution's gate, not the feed's. Both match on the `Chain` variant and fall back to a raw chain id only for a chain Tycho cannot name (Bebop's Optimism and HyperEVM, reachable by registering a `Chain::Custom`), so neither retypes tycho-common's table. Hashflow, Liquorice and Metric are addressed by a chain id alone. A list of ours could only repeat what the venue already knows, and when it drifted it would refuse a chain the venue does price, so the builder resolves the id and the venue itself answers. Each client recognises its venue's refusal in the body of the failing response and returns `Fatal`, which ends the feed and withdraws its books on the first poll. `snapshot_feed::http` carries that verdict as a `RefusalCheck` every fetch takes, so what a failing status means stays the venue's to say. The refusal has to be read from the message: Hashflow answers an unknown market maker with the same error code as an unserved chain, and a maker can vanish between the market-makers call and the price-levels one; Metric answers an unknown pool with the same 404 as an unserved chain. Hashflow's and Metric's checks therefore parse the venue's error body and read one field of it, which a body the venue does serve cannot be parsed into at all. Metric needs one guard for that: it answers an empty bearer token with the same `Unknown chainId` as an unserved chain, where a key it rejects is a plain 401. A feed built without a key would read its own misconfiguration as a refusal and end, so the builder turns an empty key away. Measured against the live APIs, which is where every chain list here comes from: Hashflow prices 1, 10, 56, 137, 8453, 42161 and 43114 and refuses the rest with `Invalid chainId`; Liquorice names its own set when it refuses one; Metric serves ten chain ids and answers `Unknown chainId` elsewhere, which narrows it from any chain whose id resolves; Native serves five and refuses everything else with `131005 chain parameters invalid`; Bebop publishes its eight at `api.bebop.xyz/pmm/chains`. The table tests of the chain mapping are gone, as the review asks: their expectations were the match arms they tested. What replaces them asks the venue — an ignored test per enum, Bebop's draining its published list so a chain it adds is heard about too — and the two builders get the `build()` test the review suggested, asserting `is_fatal()` rather than the message, which `FeedError` says carries no promise. The solver page's Chains column was the set Tycho can execute on and is now what each venue prices, with a line saying where execution is gated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bebop's and Metric's builders assembled the venue's URLs and handed the
finished strings to a client that only stored them, so the one place that
knows a venue's paths was the one place that is not about talking to it. Both
clients now take the chain and build what they request from it: Bebop the
quote and pricing endpoints under `api.bebop.xyz/pmm/{chain}/v3`, Metric
everything under `{base_url}/public/v1/evm/{chain_id}`, which also retires the
`metadata_endpoint` parameter the review found repeated at all three call
sites.
`BebopClient` has one constructor again. The second existed for tests, and
most of them did not need it: every source fixture passed a pricing endpoint
no test read, and the state fixture passed the same URLs the chain produces.
What is left is the quote retry tests, which point at a local server, and they
say so in one field beside `new`.
Metric's base URL is the only one a caller supplies, so `build()` parses it:
a URL that is not one — a missing scheme, say — fails every poll alike, and
reqwest reports it per request, which the loop retries forever. It is
`InvalidInput` at build instead. The parsed `Url` is the client's argument so
the type carries the requirement, but no field holds one: the endpoints stay
strings, since `Url::join` on a prefix like `/evm/1` silently drops the chain
id, and the four RFQ clients serialize theirs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`test_live_metric_api_state_get_amount_out` builds its own metadata request and sent no bearer token, which Metric now answers with `404 Unknown chainId` for a chain it does serve, so the test failed on a decode error. It sends the key it already reads for the bid/ask call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The source built one state per market maker and pair but keyed each under the pair's component id, so a pair several makers quote kept whichever maker the `HashMap` iteration reached last — a different one per poll, with the rest of the venue's liquidity dropped. A swap settles one signed quote from one maker, so a pair's makers cannot be summed into one book either. Each maker's ladder is its own component instead: the id is the pair's id followed by the maker's name, under the protocol system the feed publishes, so a fallback feed stamps its own. The binding quote goes to that maker alone — `marketMakers: [maker]` with `options.doNotRetryWithOtherMakers` — and the API no longer substitutes a maker whose price the state never simulated. Pinning makes a decline ordinary, because the maker asked is the only one that can answer. Hashflow states it inside an HTTP 200, and both codes it declines with ended the component: 82 `No maker supports this request` and 76 `Exceeds supported amounts` were fatal, which tells a consumer to stop simulating a book over one trade the maker would not take at that size. Both are `QuoteNotFound` now. The venue publishes no catalogue of its codes, so anything else stays fatal: an unrecognised failure is not evidence that another attempt would fare better. The live quote test found that for itself — pinned to whichever maker the map yielded first, it asked for one WETH and was declined. It picks the deepest maker whose own ladder covers the amount now, which is what a state does. Every Hashflow component id changes again, by the maker's name. Ported from #1495, which is based on the branch this PR replaces. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every `RFQError` arrived at the caller as `SimulationError::FatalError`, whose contract is that there is a bug with this pool or protocol and simulation is not to be attempted again. A connection that did not hold, a body that did not parse and a maker declining one trade are none of those: the same state quotes again at another size or another moment, and only the venue saying the component itself is unusable means what the variant claims. Each variant lands where it belongs now. The message crosses unchanged, with the prefix its variant gives it: `From<reqwest::Error>` puts a bare transport message into `ConnectionError`, which on its own names neither the venue nor the request. Nothing acts on the difference yet — tycho-execution carries it into `EncodingError` variant for variant, and the consumers flatten that into one string — but the verdict a venue gives now reaches them instead of being erased one call later. Native classifies its documented codes into temporary failures and liquidity refusals, and Hashflow tells a declined trade from a request it cannot read; both were worth nothing to a caller that saw every one of them as a broken component. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Production error counts put three refusals above every other quoting failure, and each reached the caller as fatal — which says the component has a bug and is not to be simulated again, where all three are a venue answering one request. Bebop mapped every error code to a fatal error. Code 102 `InsufficientLiquidity` and code 104 `MinSize` are the venue declining the trade at the size asked for, past its depth or below its minimum, so both are `QuoteNotFound`; a code it has not been seen to decline with stays fatal, because Bebop publishes no catalogue to read one against. Hashflow and Liquorice answered a non-200 quote response with a fatal error too, in the same block copied between them. The venue answering at all means the request reached it, so the status is about this attempt: a rate limit, a gateway in between, or a key to be fixed at the venue while the feed keeps running. It is a connection error now. Both read the status and then dropped it, logging the body alone; it is in the message with the body. Bebop's own non-200 path was already classified this way. Where a code list is what a venue is judged by, what is known about it is recorded beside the codes: Bebop's reference documents an HTTP 422 for the quote endpoint and nothing else, and Hashflow's numbered vocabulary belongs to its market-maker WebSocket API rather than the taker endpoints, which is why one of its codes covers three conditions and telling them apart needs the message. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Code 79, `Below minimum amount`, is what a request addressed to one maker gets back when it asks for less than that maker's first level. It was not among the codes read as a declined trade, so it reached the caller as fatal — the component has a bug and is not to be simulated again — for a size the same ladder quotes one level up. Measured against the live API: pinned to a maker quoting WETH/USDC, a request below its published first level comes back as 79, and 82 comes back only for a maker that publishes no levels for the pair at all. A maker without levels for a pair gets no book, so no component can name one, which leaves 79 as the refusal a pinned request meets in practice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bebop refuses a trade worth less than a dollar on the side being sold, and
nothing we published said so: `get_amount_out` went from the direction check
straight to the fill, so a leg worth cents simulated clean and the refusal only
arrived at encode time, when the firm quote came back `MinSize: Request quotes
over $1.` and the quote was lost. In a deployment without this it is 0.62 % of
quote requests. Hashflow is the contrast — it has held callers to its maker's
first level all along, which is why it has no such row.
The floor is measured rather than guessed. Bisecting the refusal against the
live API gives one dollar of the sell side everywhere — Ethereum and Base, both
directions, WETH/USDC, USDC/WETH and WBTC/USDC — and `/pmm/{chain}/v3/tokens`
publishes a `priceUsd` per token whose values reproduce every threshold. The
venue publishes no per-token minimum, so a constant and a price are the rule.
The price is one the source already works out: a book's TVL is measured in USD
through the pair that prices its quote token in a USD quote token, and a book
with no such pair is not published at all, so every book that exists can state
its floor. The state carries it as `usd_per_quote_token`, which is one more
field in its serialized form. Both numbers now come out of one lookup of that
pair, where the TVL and the floor had been searching the update for it
separately — they are still priced at different sizes, since a book's TVL is
worth what converting all of it would fetch and a dollar is a trade at the top
of the ladder.
`get_amount_out` reads the trade's worth on the quote side, which the swap
already expresses: the amount sold when the quote token is sold, and what the
ladder pays otherwise. That second direction values the trade at the maker's
own price rather than at mid, so the floor sits half a spread high; pricing it
at mid would cost another ladder walk on every quote to move a boundary the
venue's own price feed does not agree with to the basis point anyway.
Below the floor is `RecoverableError`, as Hashflow's minimum is, and not
`InvalidInput`: a consumer reads that one as an amount beyond what the book can
serve and remembers that every larger amount fails too, which is the opposite of
what a minimum says.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every source assembled a component beside its state and filed the two of them
under an id it restated: `pair_component(id, system, "<venue>_pool", chain,
base, quote)` next to a state literal carrying the same two tokens, then
`books.insert(component_id.to_string(), Book { component, state, updated_at })`.
Three of the six arguments were the same on every call a venue made, which is
the sign that they are one thing rather than six.
A state prices one pair and already holds it, so the component follows from the
state: `PairState` hands over the two tokens and `pair_book` builds the
component and the book around them. The venue's component type moves to its
module, beside the protocol system it is the other half of — it had been a
string literal in `state.rs`, two files away from the identity it belongs to.
`Books` keys each book by the component the book carries, so an id is written
where it is derived and nowhere else, and two books claiming one component warn
rather than one of them quietly disappearing — the shape of the Hashflow maker
bug, which the ids no longer allow but the collection can no longer hide.
The four HTTP sources' `fetch` is the poll itself rather than a wrapper around
one, which leaves each source a single inherent `impl` and Metric none at all.
The five states lose their public `base_token` and `quote_token`. A book's pair
is on the component next to it, in the order the venue published, which is what
the integration test reads now; it had read the state because a component's
tokens are reordered on the way through the decoder, and a book never takes
that path — it is published as the feed built it.
Venue code is 104 lines lighter and holds only decoding; what it no longer
repeats is 55 lines in `book/`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Liquorice's `use_legacy_auth` was an `Arc<AtomicBool>` inside a client that is not `Clone` and is only ever reached through an `Arc` of its own, so the inner one shared nothing with anybody. It is a plain `AtomicBool`. Liquorice's feed reached its source through `use super::source::`, where the other four name the path from the crate root. Native's `orderbook_api_error` had one caller and is the `FeedError::Connection` at that call site now. One line of its doc comment survives the move: that no failure the venue reports in an envelope is fatal, a refused key included, which is a decision a reader could otherwise undo. The rest was the error-code URL, which `classify_api_error` carries twenty lines below, and a restatement of the variant being built. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A state had three ways to say no and used them for overlapping reasons: an empty ladder was a recoverable failure at one venue and zero limits at another, a zero amount was refused by whichever floor happened to see it first or not at all, and the partial-fill error was handed out for books that could fill nothing. Each case now has one answer, the same from every venue. An amount of zero is refused before anything else looks at it. Every venue refuses one in its own words, measured against the live APIs: Bebop with `101 InvalidApiRequest: Sell amounts must be higher than 0.`, Hashflow with `79 Below minimum amount`, Liquorice with `invalid_request: Token amount is less than or equal to zero`, Native with `131004 invalid parameter`, and for Metric the pool itself, whose quoter reverts `InvalidAmount()` while the venue's routing API answers `400 amountIn must be positive`. The check and its error are shared; the comment at each call site records what that venue answers. Without it a swap of nothing read as one that worked: a ladder reports a zero amount as fully consumed, so Liquorice answered `Ok(0)` for a populated book and Metric reported a zero input as a complete fill in both of its branches — a route whose Metric leg reverts on chain, advertised as a filled swap. A side the venue is not quoting is refused by both methods: no limit to report, and no amount servable. `fill_result` makes that call for the four ladder venues, since a fill that took none of the amount is a side with nothing to take it with, and it says so with no partial result attached — there is no part of a swap to offer. Bebop's `get_limits` answered zero limits for such a side, which advertises a swap that cannot be made; it refuses like the others now. The partial-fill error is left meaning what it says: there was a ladder, the amount is past what it holds, here is what fits. `fill_result` takes the state by reference and clones it only when it builds a result, so the two paths that refuse outright no longer pay for one. Hashflow's and Liquorice's `valid_levels_guard` are gone with the cases that needed them. Bebop's `test_get_limits_invalid_token_pair` is `rejects_tokens_outside_the_pair`, which asks `spot_price`, `get_amount_out` and `get_limits`, as the review asks and as Hashflow and Liquorice already did: the PR moved all three to `InvalidInput`, where the old test covered the one method that was already there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A state carries its client, so whatever authenticates the feed rides into every recording of that state unless the client says otherwise. Hashflow and Liquorice say so on four fields between them and nothing asked: deleting an attribute would have put a key in the JSON with the suite still green, and the diff would have been one line disappearing from a struct. Both now round-trip the client and assert the fixture's credentials are nowhere in the serialized form. Bebop's and Native's equivalents are cut to the same shape — the credentials and one endpoint, as evidence that something useful survived — where they had compared field by field, which tests serde's derive and has to be revisited whenever a field is added. Native's asserted the deserialized key was empty, which `default` guarantees whether or not the key was written; it checks the serialized form instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Native's `builds_component_and_state_from_relay_orderbook` and Metric's `build_pair_component_and_state` built a state, handed it to `pair_book`, and asserted that what came back was that state and a component derived from it. Every expectation in them came from inside the test, the component assertions repeat what `book::component`'s own tests pin, and Native's grouping assertions repeat two tests that read the venue's orderbook. The one thing neither restated is the component type a venue publishes under, so that assertion moves to each venue's end-to-end test, where it rides the path a poll takes: Native's already read the component's tokens in the published order, Metric's its id. Their fixtures and imports go with them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Metric's models explained what becomes of two fields once a state is rebuilt from component attributes, and its depth comment named the same case. Nothing rebuilds a state that way: Metric reaches a consumer as a state the feed built, with no decoder and no attribute round trip, so a reader followed those clauses to code that does not exist. Each field says what it means and when it is absent; one-sided depth is the one reason a traded side has no bins. The solver page named the two chains `usd_stablecoins_for_chain` covered when the sentence was written, and it covers five. It says what the function does instead, which is also what the function's own doc says, so the list lives in one place — the match arms that hold it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Main gained #1501 while this branch was in review: an opt-in layout where a venue publishes one component for all of its pairs, whose state marks a market maker used after a swap so one route cannot take two quotes the venue would not give. It is wired in through the two things this branch deletes — a `TryFromWithBlock` decoder per venue, and `RFQStreamBuilder::add_client` under a second stream name — and it rebuilds each client by reading credentials back out of the environment, which this branch also removed. So `rfq/` resolves to this branch's structure and the layout's stack comes out whole: four `all_pairs_decoder.rs`, four `all_pairs_state.rs`, `protocols/component.rs`, `maker_price_levels.rs`, `test_utils.rs`, the `ComponentLayout`/`stream_name()` wiring in the clients and builders, and the integration test's helpers for the `swap_directions` attribute. Every commit that conflicted in `rfq/` belongs to that stack. The layouts and the quote rules are ported onto the feed layer in the commits that follow. Everything else main changed is kept: the price-level stream's split into `tracker.rs`, `telemetry.rs` and `test_support.rs` — with this branch's two-clone cleanup re-applied where `merge_pairs` now lives — the indexer's off-worker, the Ring Swap v2 BSC runner, and the release bumps. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`LiquoriceState::get_amount_out` fills against the ladder of the maker paying the most, and then asked for a firm quote without naming it, so the venue could answer with a level from any maker — a different price on a different maker's depth from the one the route was built on. It is the defect #1495 fixed for Hashflow, where the state carries its maker and the request names it. One method decides which maker fills a swap, and both halves use it: `best_fill` returns the maker and what it pays, `get_amount_out` prices with it, and `request_signed_quote` re-runs it on the same amount to name the maker the request goes to. `process_quote_response` keeps only that maker's levels, so the venue no longer decides which maker answers. The maker is re-derived rather than remembered, so the two halves agree exactly where the quote is asked on the state the swap was simulated on, with the amount it was simulated for. Liquorice prices base into quote only, so the amount a quote re-runs the selection on is always in base units. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… rules A book feed publishes a component per pair, which is what a consumer routes through. A venue that signs a quote per trade can only sign so many for one route, and a consumer carries simulated state from one swap to the next per component — so `component_layout(ComponentLayout::AllPairs)` on the Bebop, Hashflow, Liquorice and Native builders publishes the venue as one component covering every pair it prices instead. A swap against that component goes to the book paying most among those still allowed to quote, and the state it returns records the book as spent. What the next swap is held to is the venue's `QuoteRule`: `OncePerMaker` for the venues that name their makers and let a taker pick one, `OncePerVenue` for those that pick the maker themselves. The component advertises both the rule and the directions it prices as static attributes, since its token list no longer says which pairs exist. The books behind it are the ones the per-pair layout publishes, so pricing, level validation, limits and the firm quote are the per-pair states' as before; `book::all_pairs` adds the direction index, the selection and the used set. Liquorice's per-pair state holds every maker of a pair at once, so under this layout its source splits them into a book each, which is what the rule counts. Metric keeps the per-pair layout alone: its components are pool addresses and there is no quote to decline. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A venue published as one component prices a set of pairs that its token list cannot be read as, so `process_state` takes the `swap_directions` attribute where the component carries one and samples `--max-simulations` of them. The per-pair components keep the token pairing they had. `--book-all-pairs` builds the four RFQ venues under that layout, and Hashflow's minimum trade size is read per direction, since under it the makers of a pair sit behind one state. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The solver page describes the layout, the attributes a component carries under it and the rule a route is held to; the integrator page says what a venue supplies to be published that way. The quickstart builds its RFQ venues under the layout with `--all-pairs`, which the example's loop reads unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two choices the all-pairs layout made by accident rather than on purpose, now made the way the implementation it was ported from makes them. `get_limits` picked the book that takes the most input, because it compared `(sell, buy)` pairs and the sell amount came first; it now picks the book that buys the most, which is the book worth routing to. Hashflow's floor is the smallest of its makers' first levels, so an amount under it is one no maker fills. The error named whichever book answered first, whose floor can be higher than the one that would have taken the trade; it now names the floor a caller can act on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Hashflow and Liquorice name their market makers and let a taker pick one, so their `all_pairs(rule)` takes how often one route may quote them: once per maker, which is what their independence buys, or once for the venue where that independence is not worth relying on. Bebop and Native pick the maker behind a quote themselves, so their `all_pairs()` takes no rule and settles `OncePerVenue`. The rule lives in the layout the builder settles, so a rule that nothing applies cannot be expressed: not on a venue publishing a component per pair, where the states track no quotes, and not on a venue that has no choice to offer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…s-port # Conflicts: # crates/tycho-simulation/src/pamm/protocols/metric/state.rs # crates/tycho-simulation/src/rfq/protocols/bebop/client.rs # crates/tycho-simulation/src/rfq/protocols/hashflow/client.rs # crates/tycho-simulation/src/rfq/protocols/test_utils.rs
`min_amount_in` rounded the first level's quantity up to a whole unit of the token, so on a book of one level — where that level is also the whole depth — it named one unit more than the ladder holds, and `get_limits`, which rounds the total down, named one less. No size satisfied both, and a consumer that sized its request at the minimum was quoted a partial fill. Measured against Hashflow's live USDC/USDT books: 4 refusals in 824 swaps. The level is a fraction of a unit wide and the venue signs on either side of it: a live maker quoted one, two, ten and ninety units under its published level, and refused half of it with code 79. Rounding down is therefore a size the venue takes and the book holds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
| /// The maker paying the most for `amount_in` whole base tokens, and what it pays. The quote | ||
| /// goes to this maker, so the swap is signed by the one whose ladder priced it. | ||
| fn best_fill(&self, amount_in: f64) -> Option<(&str, Fill)> { | ||
| self.prices_by_mm | ||
| .iter() | ||
| .filter_map(|(maker, price)| { | ||
| if price.levels.is_empty() { | ||
| return None; | ||
| } | ||
| Some((maker.as_str(), price.levels.fill(amount_in))) | ||
| }) | ||
| .max_by(|(_, a), (_, b)| { | ||
| a.amount_out | ||
| .partial_cmp(&b.amount_out) | ||
| .unwrap_or(std::cmp::Ordering::Equal) | ||
| }) | ||
| } |
There was a problem hiding this comment.
I agree - makes this cleaner and more consistent.
🤖 Suggestion
Both venues name their makers, and both now send the quote to the maker that priced it. Hashflow publishes one component per maker and pair. Liquorice publishes one component per pair whose state picks a maker inside: best_fill picks for the swap and the deepest maker sets get_limits. That is a second copy of the selection AllPairs::best and AllPairs::get_limits already do, and consumers see two shapes for the same kind of venue.
Publish Liquorice per maker and pair the way Hashflow does, and drop the multi-maker branch from LiquoriceState. Its all-pairs layout already builds single-maker states.
| client: Arc::new(HashflowClient::new( | ||
| chain_id, | ||
| "https://api.hashflow.com/taker/v3/rfq".to_string(), | ||
| "https://api.hashflow.com/taker/v3/price-levels".to_string(), | ||
| "https://api.hashflow.com/taker/v3/market-makers".to_string(), | ||
| self.auth_user, | ||
| self.auth_key, | ||
| self.quote_timeout, | ||
| )), |
There was a problem hiding this comment.
I like the consistency recommended here.
🤖 Suggestion
The PR says client.rs holds the venue's endpoints. Bebop and Metric do: they derive them in new from the chain and a base URL. Hashflow and Liquorice take full URLs from the builder, which pushes their constructors to seven positional arguments. Native keeps its base URL as a constant in feed.rs. The production Hashflow URLs are also pasted into four test call sites.
Use the Metric shape everywhere. new takes one base URL, the client derives each path from it, and the production base is a constant in client.rs. Tests keep injecting a mock server through the base URL.
| #[debug(skip)] | ||
| pub(super) client: Arc<BebopClient>, |
There was a problem hiding this comment.
I'm not sure if it is intentional that we cannot replay them? But this suggestion makes sense in that case.
🤖 Suggestion
The client: Arc<…> field is serialized with every state. Credentials are skipped, so a replayed state fails any quote anyway. Each deserialized Arc is a separate client with its own reqwest pool, so replay loses the pooling the PR adds. An all-pairs state of ~576 books also writes the same endpoints ~576 times.
Make the state's client an #[serde(skip)] Option<Arc<…>>, and have a missing client fail the quote. That says outright that a replayed state cannot quote, and recordings shrink.
| ComponentLayout::AllPairs(_) => { | ||
| // A Bebop book quotes both ways round, so it prices both directions of the | ||
| // pair it was published as. | ||
| let directions = vec![ | ||
| (base_token.address.clone(), quote_token.address.clone()), | ||
| (quote_token.address.clone(), base_token.address.clone()), | ||
| ]; | ||
| venue_books.push(VenueBook { maker: None, directions, updated_at, state }); | ||
| } |
There was a problem hiding this comment.
Was this an intentional decision? I think the proposed fix here makes sense if this was not a purposeful choice to stop considering if the direction can actually be quoted.
🤖 Important
A solver reads the venue's list of tradable directions and adds each one to its routing graph. Some of those directions have no liquidity behind them. Every route through one fails at simulation, and the integration test logs a failure for each one it samples. Before this change, the list only named directions the venue was actually quoting.
Root cause: Each source writes VenueBook::directions by hand, but only the state knows which sides of its book have levels.
Symptoms (each would be resolved by the fix below):
- Behavior: Bebop and Native push both orientations of every pair, whether or not that side has levels. The deleted code from
#1501added a direction only for a non-empty side, and skipped books with neither. Native routinely builds one-sided books from a single entry. Its component therefore lists directions whose only book refuses with "Pool has no liquidity on this side",crates/tycho-simulation/src/rfq/protocols/bebop/source.rs:155–163,crates/tycho-simulation/src/rfq/protocols/native/source.rs:234–243 - Architecture: Hashflow and Liquorice restate the same fact a third and fourth time, so the four sources and the states'
valid_direction_guardcan drift apart,crates/tycho-simulation/src/rfq/protocols/hashflow/source.rs:145,crates/tycho-simulation/src/rfq/protocols/liquorice/source.rs:158
Proposed fix: Add fn directions(&self) -> Vec<(Bytes, Bytes)> to PairState, returning only the sides with levels, and have AllPairs read it. Drop the directions field from VenueBook, and skip a book whose state returns none. The bug goes away because the empty-side check lives in the state. The copies go away because no source writes directions any more.
| // The trade's worth is read on the quote side, which the swap already expresses: the | ||
| // amount sold when the quote token is sold, and what the ladder pays otherwise. | ||
| let quote_amount = match direction { | ||
| SwapDirection::QuoteToBase => amount_in, | ||
| SwapDirection::BaseToQuote => fill.amount_out, | ||
| }; | ||
| let worth_usd = quote_amount * self.usd_per_quote_token; | ||
| if worth_usd < MIN_TRADE_USD { | ||
| return Err(SimulationError::RecoverableError(format!( | ||
| "Invalid token addresses: {}, {}", | ||
| token_in.address, token_out.address | ||
| "Trade is worth {worth_usd} USD, below the {MIN_TRADE_USD} USD Bebop quotes" | ||
| ))); | ||
| }; | ||
| // if sell base is true -> use bids | ||
| // if sell base is false -> use asks AND amount is in quote token so the levels need to be | ||
| // adjusted | ||
| let price_levels = if sell_base { | ||
| self.price_data.get_bids() | ||
| } else { | ||
| self.price_data | ||
| .get_asks() | ||
| .iter() | ||
| .map(|(price, size)| (1.0 / price, price * size)) | ||
| .collect() | ||
| }; | ||
|
|
||
| if price_levels.is_empty() { | ||
| return Err(SimulationError::RecoverableError("No liquidity".into())); | ||
| } | ||
|
|
||
| let amount_in = amount_in.to_f64().ok_or_else(|| { | ||
| SimulationError::RecoverableError("Can't convert amount in to f64".into()) | ||
| })? / 10f64.powi(token_in.decimals as i32); | ||
| let (amount_out, remaining_amount_in) = self | ||
| .price_data | ||
| .get_amount_out_from_levels(amount_in, price_levels); | ||
| let res = GetAmountOutResult { | ||
| amount: BigUint::from_f64(amount_out * 10f64.powi(token_out.decimals as i32)) | ||
| .ok_or_else(|| { | ||
| SimulationError::RecoverableError("Can't convert amount out to BigUInt".into()) | ||
| })?, | ||
| gas: BigUint::from(70_000u64), // Rough gas estimation | ||
| new_state: self.clone_box(), // The state doesn't change after a swap | ||
| }; | ||
|
|
||
| if remaining_amount_in > 0.0 { | ||
| return Err(SimulationError::InvalidInput( | ||
| format!("Pool has not enough liquidity to support complete swap. input amount: {amount_in}, consumed amount: {}", amount_in-remaining_amount_in), | ||
| Some(res))); | ||
| } |
There was a problem hiding this comment.
Accurate error triggering is vital.
🤖 Suggestion
The MIN_TRADE_USD check runs before sim::fill_result. Selling base into a book with no bids fills nothing, so fill.amount_out is 0 and the swap fails with "Trade is worth 0 USD, below the 1 USD Bebop quotes". That is a RecoverableError about size. The other venues return the "no liquidity on this side" InvalidInput here, which is the single answer commit a698cd1 set out to give for this case.
A caller asks Bebop to sell a token on a pair where nobody is bidding. The error says the trade is too small. The caller raises the amount and gets the same error, at any size.
Check fill.remaining_in == amount_in first, or run the dollar floor only when fill.amount_out > 0.0, so an empty side reaches fill_result and gets its no-liquidity error.
| RFQError::ConnectionError(_) | | ||
| RFQError::ParsingError(_) | | ||
| RFQError::QuoteNotFound(_) => SimulationError::RecoverableError(message), |
There was a problem hiding this comment.
Agreed - let's be more precise about how these errors should be presented to be handled. Will an immediate retry guarentee the same result? If so, it's not a recoverable error.
🤖 Suggestion
From<RFQError> for SimulationError now maps ParsingError to RecoverableError. Native reports a token, recipient, signed-amount or payable-value mismatch as RFQError::ParsingError, so these come out as RecoverableError. Bebop, Hashflow and Liquorice report the same checks as FatalError. Before this change every RFQError became FatalError, so all four venues agreed.
Native signs a quote for the wrong recipient or the wrong token. The caller is told the failure is temporary, so it retries the venue. Bebop, Hashflow or Liquorice returning the same bad quote is reported as permanent.
Return RFQError::FatalError from Native's validation mismatches in process_quote_response. Keep ParsingError for bodies that fail to parse.
| // Liquorice reports `updatedAt` per market maker in milliseconds; the book is as | ||
| // fresh as its most recently updated maker. | ||
| let updated_at = mm_prices | ||
| .values() | ||
| .filter_map(|price| price.updated_at) | ||
| .max() | ||
| .and_then(|millis| DateTime::from_timestamp_millis(millis as i64)); |
There was a problem hiding this comment.
Consistency.
🤖 Suggestion
updated_at is the max of the makers' updated_at, but LiquoriceState::get_amount_out prices against whichever maker pays most, fresh or not. The all-pairs layout takes the min on purpose, so a component "is no fresher than the stalest book behind it" (crates/tycho-simulation/src/book/all_pairs.rs:196–206). The same makers therefore report different ages under the two layouts.
An operator drops books older than a minute. A pair where one maker repriced just now and another went quiet an hour ago still looks fresh. Swaps on that pair can be priced off the hour-old maker.
Use .min() over the makers' timestamps, matching AllPairs::updated_at. Only use the freshest maker if the state stops quoting makers that are older than it.
| let bid_ask = match bid_ask { | ||
| Ok(bid_ask) => bid_ask, | ||
| Err(e) => { | ||
| warn!(pool = %pool.pool_address, error = %e, "skipping pool, bid/ask fetch failed"); | ||
| continue; | ||
| } | ||
| }; |
There was a problem hiding this comment.
All-pool failures should be handled as a fail.
🤖 Suggestion
A failed per-pool fetch_bid_ask is logged and skipped. When every pool fails (a revoked key gets a 401 on each one), fetch returns Ok with no books. The poll loop counts that as a success: it clears the failure streak and replaces the live snapshot with an empty one, and max_consecutive_failures never trips. SkipInvalidEntries refuses to let the other venues do exactly this ("a response that reads as nothing withdraws every pair").
An operator's Metric key gets revoked. The feed keeps running and keeps saying it has data, but it serves no pools. Nothing reports a failure, so it looks like Metric simply has no liquidity.
Fail the poll with FeedError::Connection when quotable was non-empty and every bid/ask request failed, so the failure budget and withdrawal apply. A mock server answering 401 on every bid/ask request would pin it in a test.
Scope: The deleted Metric client did the same, but this PR adds the failure budget and the empty-response rule this path skips.
| /// Disable the RFQ book feeds (Bebop, Hashflow, Liquorice, Native) | ||
| #[arg(long, default_value_t = false)] | ||
| disable_rfq: bool, | ||
| disable_rfq_feeds: bool, | ||
|
|
||
| /// Stream RFQ venues as one component for all token pairs where the client supports it. A | ||
| /// swap then limits how often the route may quote the venue again. One component per pair by | ||
| /// default. | ||
| /// Disable the pAMM book feeds (Metric) | ||
| #[arg(long, default_value_t = false)] | ||
| rfq_all_pairs: bool, | ||
| disable_pamm_feeds: bool, | ||
|
|
||
| /// Run PAMM RFQ protocols. | ||
| #[arg(long, default_value_t = true)] | ||
| run_pamm_protocols: bool, | ||
| /// Publish each RFQ venue as one component for every pair it prices, which a route may | ||
| /// take one quote from, rather than one component per pair. The number of directions | ||
| /// simulated per venue and update is --max-simulations. | ||
| #[arg(long, default_value_t = false)] | ||
| book_all_pairs: bool, |
There was a problem hiding this comment.
Honestly I'd prefer we do not change the interface for something as minor as this... "book" is an action as well as a noun so makes names using it sometimes seem ambiguous...
🤖 Suggestion
--disable-rfq became --disable-rfq-feeds, --rfq-all-pairs became --book-all-pairs, and --run-pamm-protocols is gone. Clap rejects an unknown flag at startup. So a cluster job still passing --disable-rfq won't start the integration test at all, rather than just running with different settings. Nothing in this repo passes them, but the cluster configs live elsewhere 👀
Have you checked the deployed integration-test args? If they use the old flags, can we update them in lockstep with this release, or mention the rename in the PR's breaking-changes list so whoever deploys knows?
|
|
||
| ### Venue components | ||
|
|
||
| By default, Hashflow, Liquorice, Bebop and Native stream one component per token pair, and a swap leaves the state unchanged. `.component_layout(ComponentLayout::AllPairs)` on the client builder makes the client stream one component for all token pairs instead. Register an all-pairs client with `HashflowAllPairsState`, `LiquoriceAllPairsState`, `BebopAllPairsState` or `NativeAllPairsState` under the name `<venue>_all_pairs`, for example `.add_client::<HashflowAllPairsState>("hashflow_all_pairs", Box::new(client))`. The two layouts stream under different names, so one stream builder can carry a per-pair and an all-pairs client of the same venue. `ComponentLayout` and `QuoteRule` are in `tycho_simulation::rfq::models`. | ||
| By default every feed publishes one component per token pair, and a swap against it leaves the state unchanged. `.all_pairs()` on the Bebop and Native builders, and `.all_pairs(rule)` on Hashflow's and Liquorice's, publishes the venue as a single component covering every pair it prices instead, which is what lets a route stay within the number of quotes the venue will sign for it. Only the two venues that name their market makers take a rule, because only there is there a choice to make; `QuoteRule` lives in `tycho_simulation::book::all_pairs`. Metric has no quote to sign and keeps one component per pool. |
There was a problem hiding this comment.
Yes - this docs claim is not consistent with Hashflow's design.
🤖 Important
In the per-pair layout, the Hashflow source keys each component by pair and market maker (maker_component_id in crates/tycho-simulation/src/rfq/protocols/hashflow/source.rs:68–75). A pair three makers quote now arrives as three components. On main they collapsed into one. A consumer who reads this page and keys its graph by token pair will see several components for one edge, and nothing on the page tells them so.
Say that Hashflow's default layout publishes one component per market maker and pair. The builder doc at crates/tycho-simulation/src/rfq/protocols/hashflow/feed.rs:101 already says so. Add it to the breaking changes the release notes list.
| match self.component_layout { | ||
| ComponentLayout::PerPair => books.insert(pair_book( | ||
| self.protocol_system, | ||
| PROTOCOL_TYPE, | ||
| self.book_config.chain, | ||
| component_id, | ||
| state, | ||
| updated_at, | ||
| )), | ||
| ComponentLayout::AllPairs(_) => { | ||
| // A Bebop book quotes both ways round, so it prices both directions of the | ||
| // pair it was published as. | ||
| let directions = vec![ | ||
| (base_token.address.clone(), quote_token.address.clone()), | ||
| (quote_token.address.clone(), base_token.address.clone()), | ||
| ]; | ||
| venue_books.push(VenueBook { maker: None, directions, updated_at, state }); | ||
| } | ||
| } | ||
| } | ||
| if let ComponentLayout::AllPairs(rule) = self.component_layout { | ||
| if !venue_books.is_empty() { | ||
| books.insert(all_pairs_book( | ||
| self.protocol_system, | ||
| PROTOCOL_TYPE, | ||
| self.book_config.chain, | ||
| BebopAllPairsState::new(venue_books, rule), | ||
| )); | ||
| } | ||
| } |
There was a problem hiding this comment.
🤖 Suggestion
Every source matches on ComponentLayout, ignores the rule in the AllPairs(_) arm, then matches again after the loop to read it and build the venue component. That is four copies of the layout rule in a PR whose point is ending per-venue copies. A fifth venue, or a change to how the all-pairs book is assembled, has to land in four places.
Put a small collector in book/all_pairs.rs that the source builds from its layout. It takes each per-pair state with its id, maker and updated_at, and a finish() returns Books. The sources then never name ComponentLayout. A smaller step is to have all_pairs_book take the venue books and the rule and return Option<Book>, which turns each ten-line tail into one line.
| pub struct BebopAllPairsState(AllPairs<BebopState>); | ||
|
|
||
| impl BebopAllPairsState { | ||
| pub(super) fn new(books: Vec<VenueBook<BebopState>>, rule: QuoteRule) -> Self { | ||
| BebopAllPairsState(AllPairs::new(books, rule)) | ||
| } | ||
| } | ||
|
|
||
| impl AllPairsState for BebopAllPairsState { | ||
| type Pair = BebopState; | ||
|
|
||
| fn pairs(&self) -> &AllPairs<BebopState> { | ||
| &self.0 | ||
| } | ||
|
|
||
| fn with_pairs(&self, pairs: AllPairs<BebopState>) -> Self { | ||
| BebopAllPairsState(pairs) | ||
| } | ||
| } | ||
|
|
||
| #[typetag::serde] | ||
| impl ProtocolSim for BebopAllPairsState { | ||
| fn fee(&self) -> f64 { | ||
| 0.0 | ||
| } | ||
|
|
||
| fn spot_price(&self, base: &Token, quote: &Token) -> Result<f64, SimulationError> { | ||
| self.0 | ||
| .spot_price(&base.address, "e.address) | ||
| } | ||
|
|
||
| fn get_amount_out( | ||
| &self, | ||
| amount_in: BigUint, | ||
| token_in: &Token, | ||
| token_out: &Token, | ||
| ) -> Result<GetAmountOutResult, SimulationError> { | ||
| self.amount_out(amount_in, &token_in.address, &token_out.address) | ||
| } | ||
|
|
||
| fn get_limits( | ||
| &self, | ||
| sell_token: Bytes, | ||
| buy_token: Bytes, | ||
| ) -> Result<(BigUint, BigUint), SimulationError> { | ||
| self.0.get_limits(sell_token, buy_token) | ||
| } | ||
|
|
||
| fn as_indicatively_priced(&self) -> Result<&dyn IndicativelyPriced, SimulationError> { | ||
| Ok(self) | ||
| } | ||
|
|
||
| fn delta_transition( | ||
| &mut self, | ||
| _delta: ProtocolStateDelta, | ||
| _tokens: &HashMap<Bytes, Token>, | ||
| _balances: &Balances, | ||
| ) -> Result<(), TransitionError> { | ||
| Err(TransitionError::DecodeError("Not implemented".into())) | ||
| } | ||
|
|
||
| fn clone_box(&self) -> Box<dyn ProtocolSim> { | ||
| Box::new(self.clone()) | ||
| } | ||
|
|
||
| fn as_any(&self) -> &dyn Any { | ||
| self | ||
| } | ||
|
|
||
| fn as_any_mut(&mut self) -> &mut dyn Any { | ||
| self | ||
| } | ||
|
|
||
| fn eq(&self, other: &dyn ProtocolSim) -> bool { | ||
| self.same_books(other) | ||
| } | ||
| } | ||
|
|
||
| #[async_trait] | ||
| impl IndicativelyPriced for BebopAllPairsState { | ||
| async fn request_signed_quote( | ||
| &self, | ||
| params: GetAmountOutParams, | ||
| ) -> Result<SignedQuote, SimulationError> { | ||
| self.quoting_book(¶ms.amount_in, ¶ms.token_in, ¶ms.token_out)? | ||
| .request_signed_quote(params) | ||
| .await | ||
| } | ||
| } |
There was a problem hiding this comment.
🤖 Suggestion
Each venue's all-pairs wrapper is a copy of one block. It holds a constructor, the AllPairsState impl, ten ProtocolSim methods and IndicativelyPriced. Only the type names differ, plus Hashflow's minimum-amount check. Diffed with the venue names stripped, the Bebop, Liquorice and Native blocks are identical. That is about 360 lines, and a fix to one copy will not reach the other three.
typetag needs concrete types, but a macro_rules! can emit them. Have it take the wrapper and pair type and generate all four impls. impl_non_serializable_protocol in crates/tycho-simulation/src/serde_helpers.rs is the precedent. Give Hashflow's floor a hook, e.g. a defaulted AllPairsState::check_amount, or let Hashflow keep a hand-written get_amount_out.
| // Hashflow and Liquorice price one direction, which a book's component carries as the | ||
| // order of its two tokens. | ||
| None if [hashflow::PROTOCOL_SYSTEM, liquorice::PROTOCOL_SYSTEM] | ||
| .contains(&component.protocol_system.as_str()) => | ||
| { | ||
| match rfq_pair_direction(state.as_ref()) { | ||
| Some(direction) => vec![direction], | ||
| None => { | ||
| warn!("Failed to downcast state of {} component", component.protocol_system); | ||
| return HashMap::new(); | ||
| } | ||
| } | ||
| vec![(component.tokens[0].clone(), component.tokens[1].clone())] | ||
| } |
There was a problem hiding this comment.
Sounds reasonable. I'll leave it to you to decide.
🤖 Suggestion
A per-pair Hashflow or Liquorice component prices base into quote only, but its attributes say nothing about it. The integration test therefore keys on hashflow::PROTOCOL_SYSTEM and liquorice::PROTOCOL_SYSTEM. A Hashflow feed built with_fallback_router() publishes fallback:rfq:hashflow, misses that check, and gets simulated in a direction no maker quotes.
Have pair_book stamp SWAP_DIRECTIONS_ATTRIBUTE on a one-way pair, using the state's own directions. Consumers then read one attribute for both layouts and stop listing venues.
| @@ -577,29 +162,25 @@ impl RFQClient for BebopClient { | |||
| if elapsed >= self.quote_timeout { | |||
| return Err(last_error.unwrap_or_else(|| { | |||
| RFQError::ConnectionError(format!( | |||
| "Bebop quote request timed out after {} seconds", | |||
| self.quote_timeout.as_secs() | |||
| "Bebop quote request timed out after {:?}", | |||
| self.quote_timeout | |||
| )) | |||
| })); | |||
| } | |||
|
|
|||
| let remaining_time = self.quote_timeout - elapsed; | |||
|
|
|||
| let request = client | |||
| let request = self | |||
| .http | |||
| .get(&url) | |||
| .query(&query) | |||
| .header("accept", "application/json") | |||
| .bearer_auth(&self.ws_key); | |||
| .bearer_auth(&self.key); | |||
|
|
|||
| let response = match timeout(remaining_time, request.send()).await { | |||
| Ok(Ok(resp)) => resp, | |||
| Ok(Err(e)) => { | |||
| warn!( | |||
| "Bebop quote request failed (attempt {}/{}): {}", | |||
| attempt + 1, | |||
| MAX_RETRIES, | |||
| e | |||
| ); | |||
| warn!(attempt = attempt + 1, max_attempts = MAX_RETRIES, error = %e, "quote request failed"); | |||
| last_error = Some(RFQError::ConnectionError(format!( | |||
| "Failed to send Bebop quote request: {e}" | |||
| ))); | |||
| @@ -611,8 +192,8 @@ impl RFQClient for BebopClient { | |||
| } | |||
| Err(_) => { | |||
| return Err(RFQError::ConnectionError(format!( | |||
| "Bebop quote request timed out after {} seconds", | |||
| self.quote_timeout.as_secs() | |||
| "Bebop quote request timed out after {:?}", | |||
| self.quote_timeout | |||
| ))); | |||
| } | |||
| }; | |||
| @@ -623,12 +204,7 @@ impl RFQClient for BebopClient { | |||
| { | |||
| Ok(resp) => resp, | |||
| Err(e) => { | |||
| warn!( | |||
| "Bebop quote response parsing failed (attempt {}/{}): {}", | |||
| attempt + 1, | |||
| MAX_RETRIES, | |||
| e | |||
| ); | |||
| warn!(attempt = attempt + 1, max_attempts = MAX_RETRIES, error = %e, "quote response parsing failed"); | |||
| last_error = Some(RFQError::ParsingError(format!( | |||
| "Failed to parse Bebop quote response: {e}" | |||
| ))); | |||
| @@ -648,475 +224,177 @@ impl RFQClient for BebopClient { | |||
| RFQError::ConnectionError("Bebop quote request failed after retries".to_string()) | |||
| })) | |||
| } | |||
There was a problem hiding this comment.
🤖 Suggestion
The PR body cites Hashflow's and Liquorice's retry loops as its example of copies that drifted. Both are kept, along with Bebop's, and this diff rewrites the same log lines and status handling in each copy. The deadline-versus-body-read gap the PR lists as a follow-up will need fixing three times.
Extract one helper that takes the deadline, the attempt count and a closure that sends one attempt and parses it. Do it here, since every copy is already touched, or name it in the follow-up list.
| pub fn new( | ||
| chain: Chain, | ||
| tokens: HashSet<Bytes>, | ||
| tvl: f64, | ||
| ws_key: String, | ||
| quote_tokens: HashSet<Bytes>, | ||
| chain: BebopSupportedChain, | ||
| key: String, | ||
| quote_timeout: Duration, | ||
| origin_address: Option<Bytes>, | ||
| origin_target: Option<Bytes>, | ||
| origin_source: Option<String>, | ||
| ) -> Result<Self, RFQError> { | ||
| let url = chain_to_bebop_url(chain)?; | ||
| Ok(Self { | ||
| price_ws: "wss://".to_string() + &url + "/pricing?format=protobuf", | ||
| quote_endpoint: "https://".to_string() + &url + "/quote", | ||
| tokens, | ||
| chain, | ||
| tvl, | ||
| ws_key, | ||
| quote_tokens, | ||
| ) -> Self { |
There was a problem hiding this comment.
Would be cleaner
🤖 Nit
The three origin_* options always travel together from builder to client. Group them into one struct and bring the constructor down from six arguments to four.
| continue; | ||
| } | ||
| BookFeedEvent::Ended(BookFeedOutcome::Panicked(error)) => { | ||
| eprintln!("Book feed {protocol_system} died of a bug: {error}"); |
There was a problem hiding this comment.
🤖 Nit
snapshot_feed docs say "its task died" or name Panicked. Write "panicked" at these three sites.
|
|
||
| 1. Open a live feed per off-chain book venue — the RFQ market makers and the pAMMs — and read | ||
| each venue's complete current book. | ||
| 2. Leverage Tycho Simulation to get the best quoted prices across them. |
There was a problem hiding this comment.
🤖 Nit
Before: 2. Leverage Tycho Simulation to get the best quoted prices across them.
After: 2. Use Tycho Simulation to find the best quoted price across them.
Why: simplicity — "leverage" is an inflated "use".
|
|
||
| ### Venue components | ||
|
|
||
| By default, Hashflow, Liquorice, Bebop and Native stream one component per token pair, and a swap leaves the state unchanged. `.component_layout(ComponentLayout::AllPairs)` on the client builder makes the client stream one component for all token pairs instead. Register an all-pairs client with `HashflowAllPairsState`, `LiquoriceAllPairsState`, `BebopAllPairsState` or `NativeAllPairsState` under the name `<venue>_all_pairs`, for example `.add_client::<HashflowAllPairsState>("hashflow_all_pairs", Box::new(client))`. The two layouts stream under different names, so one stream builder can carry a per-pair and an all-pairs client of the same venue. `ComponentLayout` and `QuoteRule` are in `tycho_simulation::rfq::models`. | ||
| By default every feed publishes one component per token pair, and a swap against it leaves the state unchanged. `.all_pairs()` on the Bebop and Native builders, and `.all_pairs(rule)` on Hashflow's and Liquorice's, publishes the venue as a single component covering every pair it prices instead, which is what lets a route stay within the number of quotes the venue will sign for it. Only the two venues that name their market makers take a rule, because only there is there a choice to make; `QuoteRule` lives in `tycho_simulation::book::all_pairs`. Metric has no quote to sign and keeps one component per pool. |
There was a problem hiding this comment.
🤖 Nit
Before: Only the two venues that name their market makers take a rule, because only there is there a choice to make; QuoteRule lives in tycho_simulation::book::all_pairs.
After: Only Hashflow and Liquorice take a rule, because only they name their market makers. QuoteRule lives in tycho_simulation::book::all_pairs.
Why: clarity — names the two venues.
| - **`snapshot_feed/`**: latest-value snapshot feeds, both ends. `SnapshotFeed` is the contract — | ||
| `run(publisher)` consumes the source and publishes through a `Publisher<Self::Snapshot>` until | ||
| it ends with `Result<(), Self::Error>`. A feed cannot make a `Publisher`: the constructor is | ||
| crate-private and its only callers are the three consumer types, each of which creates the | ||
| watch channel, marks its `None` seed seen, and spawns the feed in its own task — so reading a | ||
| feed is the only way to run one, the channel coalesces for a slow consumer instead of | ||
| back-pressuring the venue, and no consumer can hold a feed's future and give it their own pace. | ||
| `publishing(max_age, snapshots)` is the publisher's whole public surface, so a feed hands over | ||
| its snapshots and the age at which one goes stale and nothing else is its to get right: | ||
| `Publisher`'s `Drop` withdraws whatever it leaves behind however it ends, and a feed nobody | ||
| reads is stopped by dropping its task rather than by noticing. The consumer types all spawn | ||
| the feed — hence `spawn`, not `new` — and abort it when dropped: `SnapshotFeedStream` yields | ||
| `Published` / `Withdrawn` and, as its last item, `Ended(SnapshotFeedOutcome)` — `RanOut`, | ||
| `Failed(E)` or `Panicked(String)`, the same three the watch reports. `Ended` is an event | ||
| rather than just the stream ending because the keyed set needs it per feed: its own end comes | ||
| only once every feed has ended. `SnapshotFeedStreams` keys any number of those by label, is | ||
| itself a `Stream` of `(label, event)`, and hands a feed back from `add` when the label is | ||
| taken; and | ||
| `SnapshotFeedWatch` hands out `watch::Receiver` clones for a consumer that prices on demand, | ||
| with `ended()` for the `SnapshotFeedOutcome` a bare `None` in the channel cannot express — | ||
| reported once, then pending forever, so it can sit in a `select!` arm. The trait's | ||
| associated types carry the bounds every consumer needs (`Snapshot: Send + Sync + 'static`, | ||
| `Error: Send + 'static`), so a feed that compiles can be run. The trait and the streams are generic in both the snapshot and the error; behind the | ||
| `book-feeds` feature come the feeds this crate ships, with `snapshot_feed::errors::FeedError` as their | ||
| error and the transport loops that build one from a provider's source: | ||
| `snapshot_feed::ws::run_ws_feed` drives a `WsSource` (`request` + `decode`), | ||
| `snapshot_feed::http::run_http_poll_feed` an `HttpSource` (`fetch`), each with its tuning type beside it | ||
| (`WsFeedConfig`, `HttpFeedConfig`; no `Default` — spread from the builder's `default_feed_config()`). | ||
| The loops know nothing of a venue at all: the `snapshot_feed{provider}` span their events are | ||
| recorded in is raised by `SnapshotFeedStream::spawn(provider, feed)` from the name the consumer | ||
| reads the feed under, so a venue is spelled once. They publish what the source yields, count | ||
| failures and back off (`failures.rs`), and withdraw a snapshot that goes `max_snapshot_age` | ||
| without a refresh (`publisher.rs`, which also withdraws whatever a feed leaves behind when it | ||
| ends) |
There was a problem hiding this comment.
🤖 Suggestion
A 9-line map entry became one paragraph that argues the design ("nothing else is its to get right"). An agent reading the guide to find where code lives has to read 34 lines to learn the file layout. The rationale already lives in the snapshot_feed module docs.
Keep the bullet to what lives where (SnapshotFeed, Publisher, the three consumer types, the two loops and their configs), one sentence each. Leave the reasons in the module docs.
| with `ended()` for the `SnapshotFeedOutcome` a bare `None` in the channel cannot express — | ||
| reported once, then pending forever, so it can sit in a `select!` arm. The trait's | ||
| associated types carry the bounds every consumer needs (`Snapshot: Send + Sync + 'static`, | ||
| `Error: Send + 'static`), so a feed that compiles can be run. The trait and the streams are generic in both the snapshot and the error; behind the |
There was a problem hiding this comment.
🤖 Nit
Edits were spliced into wrapped lines without rewrapping: line 57 is 148 characters, while the book/ bullet at lines 82–86 breaks after 28. Rewrap the touched paragraphs at 100.
The second of three PRs replacing #1440, and the one with the behaviour in it: #1554 is merged, and #1562 follows this one.
What is already merged
#1554 added the feed layer and nothing that runs: the
SnapshotFeedtrait, thePublisheran implementor is handed and cannot construct itself, a WebSocket loop and a polling loop, three ways to read a feed (BookFeedStream,BookFeedStreams,BookFeedWatch), and thebook/vocabulary — books, snapshots, price ladders, TVL conversion, component ids. Nothing implemented the trait, so no venue changed. None of that surface changes here.What this PR does
Bebop, Hashflow, Liquorice, Native and Metric are ported onto that layer, and the path they used before — five per-protocol decoders, the wire-format round trip through
TychoStreamDecoder, and theRFQStreamBuildermerge layer — is deleted.1011 keepalive ping timeouta customer's Bebop connection kept closing with.TychoStreamDecodercould parse it back in the same process, which also announced every component as new on every update: 23–39 % less CPU, and ~576 re-additions per update become 0.all_pairs()on a venue's builder publishes it as one component for all of its pairs, held to aQuoteRule, with the directions it prices on the component. feat(simulation): RFQ venues as one component for all pairs with quote reuse rules #1501 merged while this was in review and wired that layout through the decoders and the client builder this PR deletes, so it is ported rather than merged.What breaks, and what someone has to do
RFQClient, every*Client/*ClientBuilderandRFQStreamBuilderare gone. BuildBebopFeedBuilder::new(book_config, usd_quote_tokens, ws_key).build()?— the config first, credentials as arguments. Thetokens(),tvl_threshold(),quote_tokens()andpoll_time()setters, and Native'sfrom_env(), go with them.BookFeedStreamsand read(provider, event), hold a singleBookFeedStream, or keep aBookFeedWatchyouborrow()when pricing.Withdrawnmeans stop quoting that venue — a book is taken away once nobody is refreshing it, where the last one used to stand for the length of an outage.Endedsays a provider stopped, and why; the merge task's result was dropped before, so a dead feed went unnoticed until someone saw stale prices.min_tvl_usdand the USD quote-token set (book::quote_tokens::usd_stablecoins_for_chain(chain)) are all passed in. Metric's API key is required rather than optional.pamm::protocols::metricand its state is no longerIndicativelyPriced, because v1 signs nothing. Its token map is both filter and metadata source, so pass the full map where an empty filter used to mean "every pool".HashflowState::min_amount_in()and, on the all-pairs state, the same minimum per direction. A book's pair is on the component beside it,component.tokens, in the orientation the venue published.state.as_indicatively_priced()?.request_signed_quote(params).await.Upgrading
Building a feed — before (main):
After:
Three complete consumers are in the diff:
examples/book_quickstart, the integration test'sbook_stream_processor, and — outside this repo — propeller-heads/fynd#562, which ports the consumer side and lands once this is released.Why the feeds look like this — the incident, the benchmark, the five copies
A customer's Bebop pricing socket flapped, and every flap replayed the whole book
A solver running Fynd on tycho-simulation 0.357.2 reported Bebop closing the pricing WebSocket every few minutes with
1011 keepalive ping timeout. The flapping sat on top of a cost that was already there: Bebop pushes its full book on every update (roughly 228 KB of protobuf every 1.7 s, about 130 KB/s), every update travelled through an encode → decode round trip and a bounded queue, and the stream layer reported the entire book asnew_pairson every update — 500-plus "new" components every couple of seconds, reproduced 1:1 on a release build. A consumer that applies each update under a write lock has to keep up with that or fall behind; each reconnect then re-delivered the whole book to a consumer that was already behind.The closes themselves came from the shape of the stream.
BebopClient::stream()was anasync_streamgenerator feeding that queue, so the socket was polled only when the consumer polled. Once the consumer fell behind by more than Bebop's ~20 s pong deadline, the socket sat unpolled, the pong went out late, and Bebop closed the connection. Reproduced end to end: Fynd under a 5 % CPU quota plus quote load closed with exactly that frame every 100–130 s, with RFQ lag ramping past 200 s; at 10 % it kept up.The socket now belongs to a dedicated feed task that is polled regardless of the consumer, and the consumer gets a watch channel holding the latest complete book. A stalled consumer wakes up to the newest book and nothing else; the socket never sees the stall. Regression test:
snapshot_feed::ws::tests::answers_pings_while_the_consumer_stalls(#1554).Benchmark. This work against 0.387.0, both driving Fynd against the live Bebop book on Ethereum. Release builds, slices alternating between the two sides so each sees the same market, warm-up discarded, three configurations from an idle consumer to the full on-chain protocol set pinned to two cores — the last of them run twice, three hours apart. The
--protocolsstrings below are the ones both sides were given on the day.rfq:bebopaloneall_onchain,rfq:bebopall_onchain,rfq:bebop, two coresall_onchain,rfq:bebop, two cores23–39 % less CPU for the same work — a steady 1.7 to 2.3 CPU-seconds a minute — at identical throughput and cadence: about 31 updates a minute roughly 1.95 s apart on both sides, and no book older than a second at apply time in any run. Peak RSS is 13 MB lower with only the book feed running, and within noise once an on-chain market dominates it.
Most of that is the round trip this PR deletes. On 0.387.0 the client emits a full
StateSyncMessagesnapshot per frame andTychoStreamDecoderreports everything in a snapshot as a new pair, so the consumer re-inserts all ~576 components and announces 576 additions to everything downstream, 32 times a minute, indefinitely. Here a component is announced once and later frames are state refreshes.Five integrations that had drifted into five copies of the same loop
Reviewing the Native integration (#1244) against the four existing ones showed that the shared part of an RFQ integration — connect or poll, count failures, back off, filter by TVL, build components and states, read credentials, request a quote with retries — existed five times, with five sets of defaults and defects, and that in most cases one of the five already had the correct version.
Measured on main: 58 % of the non-test lines in the five provider directories implement those shared concerns, against a shared layer that barely existed — Hashflow's and Liquorice's retry loops, to take one pair, differ only in a comment and two auth headers.
The copies did not stay in sync. In the module's history every commit touching several providers is a mechanical refactor or a lint fix; not one behavioural fix was ever applied across providers, and ten fixes landed in exactly one copy while the siblings that need them still lack them.
Adding Native cost some 3,600 lines under
rfq/: its builder and its decoder are near-verbatim copies of Hashflow's and Liquorice's, and over half of it is tests re-testing shared-shape behaviour.Fixed here, because they are the shape of the loop rather than venue behaviour:
run_ws_feedand onerun_http_poll_feednow drive every provider, with oneFailureTracker, oneWsFeedConfig/HttpFeedConfig, a request timeout on every poll, and a rejected zero interval (previously guarded by Native alone; tokio's interval panics on zero).chain,tokens,tvl_thresholdandquote_tokenswith its own setters and defaults; Metric's TVL floor defaulted to 0, the others to 100. OneBookFeedConfig, taken by value by every builder.reqwest::Clientfor every binding quote — a full TCP and TLS handshake inside the quote deadline; Metric already pooled. Each venue now has one pooled client, shared by its feed and by every state it emits, so the book poll and the quotes reuse the same connections. Native previously held a second pool for its orderbook poll.Debugwith its key in clear text and compensated by hand-writingDebugon the state. Credentials are now skipped where they live.(price, quantity)ladder; only Native's copy guarded against non-finite results. OneLevelsnewtype that every venue decodes its wire levels into at ingestion and that guarantees finite positive quantities and prices, and onebook::simfor what every state does around it: direction detection, decimal scaling, the partial-fill result, checked limit conversion (three venues cast throughu128). Component construction, pair ids, the token-universe lookup, the TVL floor check, the send/status/parse HTTP shape and the address conversion each had a copy per venue and now have one.Left for follow-ups, because they change routing or quoting behaviour on live venues and belong in their own PRs:
spot_pricemeans three different things — the mid for Bebop, Native and Metric, the best level for Hashflow, the best whole-book average across makers for Liquorice — and only Native checks that the result is finite and positive.fee()anddelta_transition()aretodo!()in Hashflow and Liquorice.new_stateclaims an inexhaustible book: the state it hands back after a swap still holds the depth the swap consumed.HashSetiteration order, where Hashflow and Liquorice sharebook::tvland Native picks the deepest book and prices the TVL amount rather than one unit. The planned fix is that shared normalization taking Native's rule, with Bebop on it.Where repetition is legitimate and stays: WebSocket versus polling; two-sided books (Bebop, Native) versus one-directional level arrays (Hashflow, Liquorice) versus Q64.64 prices with cumulative depth (Metric); per-market-maker books for two venues only; Native's merge of mirrored orderbooks; and four irreducibly different signed-quote request shapes, Metric having none.
The RFQ path was co-opting
TychoStreamDecoderTychoStreamDecoderexists to turn indexer output — wire-formatComponentWithStatewith attribute byte maps, block headers, pending-state deltas — intoProtocolSimstates. RFQ books are neither indexed nor block-anchored, and both ends of the trip lived in one process; nothing ever crossed a wire. This was conceptually wrong from the start and only became visible when it became expensive.What it cost on main: every client serialised its bids and asks into JSON attribute bytes, and a per-protocol
decoder.rs(285 to 377 lines each, five of them) parsed them straight back; the decoder rebuilt each state's quoting client from environment variables, silently dropping the builder's configuration — credentials, Bebop origin identification, quote timeouts (the #1280/#1281 finding);Update.block_numberhad to becomeblock_number_or_timestampand aTimestampHeaderhad to impersonate a block header, which produced a real downstream bug in a consumer that evicted overlays by treating the timestamp as a block number. The merge layer on top forwardedUpdates whoseremoved_componentseach client computed against the book it had sent before. A removal is only meaningful to a consumer that applied every earlier update, so the layer had to be a queue that delivers each update exactly once — which is what tied the socket's fate to the consumer's pace. Thenew_pairsside was degenerate regardless: every client re-sent its whole book as snapshot states on every update, and the decoder reports every snapshot state as a new pair.This PR deletes the decoders, the attribute round trip,
TimestampHeader,rfq::streamand theRFQClienttrait. Feeds emit ready-to-simulate books directly.Update.block_number_or_timestampbecomes a block number again in #1562, which is only correct onceTimestampHeaderis gone — which is why it comes last.Design
#1554 has the
SnapshotFeedcontract, thePublisheran implementor is handed and cannot construct, the two transport loops, the three reader shapes, and thebook/vocabulary. What follows is what this PR adds on top of it.Module hierarchy.
rfq/holds the binding-quote family (Bebop, Hashflow, Liquorice, Native) and the crate-privateRFQErrorfor the quoting path;pamm/holds Metric, a pAMM that is priced off-chain but executed against the pool — Metric's own documentation describes pools that "quote off a live reference price instead of discovering price from their own reserves", with makers setting bid, ask and spread through a Price Provider contract ("PropAMM on demand") and trades reverting on a stale price.The line between the two is who the counterparty is: a venue is an RFQ when a maker can decline a specific trade after pricing it and signs a commitment to that one trade, and a pAMM when its pool fills any taker who arrives with fresh price data, even data fetched per trade (pre-v1 Metric attached a signed oracle update and was a pAMM all along; an
IndicativelyPricedstate is a mechanism, not the family).rfq/andpamm/depend onbook/, never on each other;FeedErrorandRFQErrorare parallel enums with no conversion between them, and the two halves never call each other: the feed path (loops, sources, builders) only producesFeedError, the quoting path (request_binding_quote, validation) onlyRFQError, and each converts intoSimulationErroron its own. That matters because the two halves have different consumers: the encoders in tycho-execution need the states and their quoting clients at encode time and have no use for WebSocket and polling machinery, so the quoting half is what a later extraction into a shared crate would move, and this split keeps that move free of feed dependencies.Per provider:
feed.rswith the public<Provider>Feedand<Provider>FeedBuilder, a privateclient.rs, a privatesource.rs, andstate.rs. The feed is{ feed_config, source }, and itsrunis one call handing each half to its loop.The line between client and source is what talks to the venue versus what interprets the answer.
client.rsholds the venue's endpoints, its credentials and one pooledreqwest::Client, and it makes every request: the book poll or the WebSocket handshake the feed needs, and — where the venue signs quotes — the binding-quote request the emitted states make through anArcto it.source.rsimplementsHttpSource(fetch) orWsSource(request+decode) and does the decoding, the orientation, the TVL measurement and the component and state building, returning aBookSnapshotstamped with the feed's clock. No source holds a credential or assembles a request.Which chains a venue prices. Where the venue's requests name the chain — Bebop in its path,
api.bebop.xyz/pmm/{chain}/v3, Native in achain=ethereumquery parameter — the list is ours: a<Venue>SupportedChainin the provider'smodels.rs, withbuild()refusing anything else. Where the API takes a chain id instead, the builder resolves the id and the venue itself answers, because a list of ours there could only repeat what the venue already knows and would refuse a chain it does price as soon as it drifted. Each of those three clients recognises its own venue's refusal in the body of the failing response, whichsnapshot_feed::httpcarries as a verdict every fetch takes, so what a failing status means stays the venue's to say. Both lists are measured against the venue's published API and held to it by an#[ignore]d test, rather than by a table that restates the match arms under it.The
snapshot_feed{provider}span a feed's logs are recorded in is raised by the consumer from the name it added the feed under, so nothing per-provider has to remember it and a venue is spelled once instead of twice with nothing keeping the two in step.One component for a venue's pairs. A consumer that carries simulated state from one swap to the next does so per component, so a route can only be held to the number of quotes a venue signs for it if the venue is one component.
all_pairs()on the four RFQ builders publishes it that way — taking the rule on the two that name their makers: the books behind it are the per-pair states the other layout would publish,book::all_pairspicks the best one still allowed to quote and records it as spent in the state the swap returns, and the component carries the rule (once_per_makerfor the venues that name their makers,once_per_venuefor those that pick them) and the directions it prices, since its token list is not its pairs. Per pair stays the default, and Metric has no quote to decline. This is #1501's feature on this PR's primitives: pricing, level validation, limits and the firm quote are the per-pair states' as everywhere else, which is why it is ~400 lines rather than the ~1 100 it replaced.Where the ported layout differs from #1501. Selection, the quote rule and its attribute, the direction encoding, the per-maker scoping and the choice of a single maker's limits are the same; four things are not, and all four follow from this PR rather than from the layout:
rfq:<venue>, where feat(simulation): RFQ venues as one component for all pairs with quote reuse rules #1501 hashes"<venue>_<chain id>". A feed serves one chain and a consumer keys its components within one, so the venue is the whole identity; a pair's id adds its two addresses for the same reason, and neither hashes what it carries.InvalidInput, notRecoverableError: it is what the per-pair states of this PR already report for a side nothing can fill, and a router reads it as "no larger amount works either".Bookon this PR does. A venue's books are filtered bymin_tvl_usdin the feed before anything is published, which is the same threshold a consumer passed to the client before (fynd passes its--min-tvlto each RFQ client today, and readscomponent_tvlnowhere).Feed defaults per venue. The feed configs have no
Default: every builder exposesdefault_feed_config()with the values that fit its venue — the shared WebSocket set for Bebop, the shared polling set for Hashflow, Liquorice and Native, and Metric's own slower one, since its poll is a page of pool metadata plus a request per pool — and a consumer spreads from that, so a venue-specific default can never be dropped by an anonymous..Default::default().The quickstart, the integration test and fynd are the three complete consumer examples.
Breaking changes, grouped and then item by item
Grouped below; the per-item list with exact signatures follows. #1554 already landed the layer's surface —
snapshot_feed::{SnapshotFeed, Publisher, SnapshotFeedStream, SnapshotFeedStreams, SnapshotFeedWatch, SnapshotFeedEvent, SnapshotFeedOutcome},snapshot_feed::errors::FeedError,snapshot_feed::ws::WsFeedConfig,snapshot_feed::http::HttpFeedConfig, and all ofbook::{Book, BookSnapshot, ReceivedAt, BookFeedConfig, BookFeedStream, BookFeedStreams, BookFeedWatch, BookFeedEvent, BookFeedOutcome}withbook::quote_tokens::usd_stablecoins_for_chain— and none of it changes here. What this PR adds to the public surface is the venues: apammmodule besiderfq, and per provider a module-levelPROTOCOL_SYSTEM,<provider>::feed::{<Provider>Feed, <Provider>FeedBuilder}andstate::<Provider>State(private fields, token accessors). The opt-in layout addsbook::all_pairs::{QuoteRule, SWAP_DIRECTIONS_ATTRIBUTE, decode_swap_directions}, the fourstate::<Provider>AllPairsState, andall_pairs()on the four RFQ builders, which takes the rule on Hashflow's and Liquorice's. What it removes is everything the old path exposed, below. Everything else underbook/,rfq/andpamm/is crate-private.RFQClient,rfq::stream,TimestampHeader, per-protocol decoders,rfq::protocols::utilsremoved;pammadded.*Client→*Feed,*ClientBuilder→*FeedBuilderwithnew(book_config, …)signatures for all five providers including Native,PROTOCOL_SYSTEMas module constants, Metric relocated topamm.pub(super); only the provider's own source builds its states), token accessors, changed serialized form; tokens outside the pair areInvalidInputeverywhere (Bebop and Native'sspot_pricereturnedRecoverableError);MetricStateno longer implementsIndicativelyPriced(its v1 impl recomputedget_amount_outwith an empty attribute map, and the Metric swap encoder never casts to the trait).FeedErrorfor the feed path;RFQErrorstays the quoting path's error but is crate-private now, since no public signature ever carried it (states convert it intoSimulationError); no conversion between the two.BookFeedConfigbecomes every builder's first argument, no public default constants, no environment reads, auth helpers andBebopOriginsremoved.derive_morebecomes a direct dependency;serde_withis a new optional dep ofbook-feeds(both were already in the lock; only the edge is new).Per-item list — every API-visible change, with signatures.
A. Streaming surface
rfq::client::RFQClientremoved (stream(),request_binding_quote()). Replacement:tycho_simulation::snapshot_feed::SnapshotFeed, added in feat(snapshot-feed): add the latest-value snapshot feed layer #1554. Binding quotes go throughIndicativelyPricedon the emitted states.rfq::streamremoved, with it the only public item it held,RFQStreamBuilder. Consumers read feeds throughSnapshotFeedStream,SnapshotFeedStreamsorSnapshotFeedWatch(feat(snapshot-feed): add the latest-value snapshot feed layer #1554);book::{BookFeedStream, BookFeedStreams, BookFeedEvent, BookFeedWatch, BookFeedOutcome}are the same with the book types filled in. Producer-sidenew_pairs/removed_pairsdiffs no longer exist — every snapshot is complete, so diffing is consumer bookkeeping.rfq::models::TimestampHeaderremoved. A feed publishesbook::BookSnapshot<A> { anchor: A, books: Arc<HashMap<String, Book>> }withReceivedAt(pub DateTime<Utc>)on the feed's clock (no accessor; readanchor.0), andbook::Book { component: ProtocolComponent, state: Arc<dyn ProtocolSim>, updated_at: Option<DateTime<Utc>> }is one pair's book.updated_atis the provider's clock:Somefor Bebop (last_update_ts), Liquorice (newest maker'supdatedAt) and Metric (server_ts),Nonefor Hashflow and Native, whose payloads carry no timestamp.decodermodules removed (rfq::protocols::{bebop,hashflow,liquorice,metric,native}::decoder); feeds emit ready-to-simulate states directly.rfq::protocols::utilsremoved.default_quote_tokens_for_chainisbook::quote_tokens::usd_stablecoins_for_chain(Chain) -> Option<HashSet<Bytes>>since feat(snapshot-feed): add the latest-value snapshot feed layer #1554 (wasResult<_, RFQError>;Nonemeans no curated set for that chain).pamm.B. Provider types
*Client→*Feedand moved to<provider>::feed:BebopFeed,HashflowFeed,LiquoriceFeed,MetricFeed, andNativeFeed(from feat: integrate native RFQ protocol #1244'snative::client::NativeClient, which was feed and quoter in one). Each implementsSnapshotFeedwithSnapshot = BookSnapshot<ReceivedAt>andError = FeedError.NativeClient::{PROTOCOL_SYSTEM, DEFAULT_ENDPOINT}→native::PROTOCOL_SYSTEM; the endpoint is private.*ClientBuilder→*FeedBuilder, in<provider>::feed(client_buildermodules gone).new()takesBookFeedConfigby value first:BebopFeedBuilder::new(book_config, usd_quote_tokens, ws_key)HashflowFeedBuilder::new(book_config, usd_quote_tokens, auth_user, auth_key)LiquoriceFeedBuilder::new(book_config, usd_quote_tokens, auth_solver, auth_key)NativeFeedBuilder::new(book_config, usd_quote_tokens, api_key)MetricFeedBuilder::new(book_config, api_key)usd_quote_tokens: impl Into<Arc<HashSet<Bytes>>>is the USD-priced set the first four normalize book TVL into; Metric's API reports USD TVL, so its builder takes none. Metric's Bearer key is a requiredString(main: anOptionread fromMETRIC_API_KEYinside the builder) — without it the bid/ask endpoint rejects every pool. Setterstokens(),tvl_threshold()andquote_tokens()are gone (configure on the config); Metric also losesquote_timeout()andapi_key(), keepingbase_url()andfeed_config().poll_time()is gone from all four HTTP builders: the cadence isHttpFeedConfig::poll_interval, set throughfeed_config()— and a call that raises it above the 30 smax_snapshot_ageshould raise that too, or the feed serves each book for 30 s and nothing until the next poll (item 24). Native losesfrom_env().build()returnsResult<*Feed, FeedError>(wasRFQError).*Client::new(...)feed constructors are gone; construct through the builders.*Clienttypes are gone from the public API. The name now denotes the crate-private client that talks to the venue (pub(crate) mod <provider>::client) — for the four RFQ venues that is also the quoting client, which their states reach throughIndicativelyPriced; Metric's is held by its feed alone. A staleuse …::client::BebopClientfails with "private module".PROTOCOL_SYSTEMmoves from associated consts to module constants:rfq::protocols::{bebop,hashflow,liquorice,native}::PROTOCOL_SYSTEM,pamm::protocols::metric::PROTOCOL_SYSTEM. Their values are unchanged:rfq:bebop,rfq:hashflow,rfq:liquorice,rfq:native,rfq:metric.MetricClient::create_component_with_stateremoved.rfq::protocols::metric::*→pamm::protocols::metric::*.MetricStateholds no client: v1'srequest_signed_quotecomputes locally and never used the one it stored, so the state no longer implementsIndicativelyPriced(as_indicatively_priced()returns the default error). The crate-privateMetricClientthat talks to the API belongs to the feed alone.C. States
*Statetypes have no constructor: fields arepub(super)and the provider's ownsource.rswrites the struct literal. All fields are private (werepub) — token identity is stored redundantly next to the raw book data and nothing re-validates it, so a field write could desynchronize direction matching, decimals or the quoting client. The only accessors areHashflowState::min_amount_in() -> BigUintandHashflowAllPairsState::min_amount_in(&Bytes, &Bytes) -> Option<BigUint>, the smallest its makers on that direction accept; a state's pair is read off theBook's component, whosetokenscarry it in the published orientation. No accessors for the raw books either, and every provider'smodelsandclientmodules are private.SimulationError::InvalidInput(_, None)by every book state'sspot_price,get_amount_outandget_limits(Bebop, and Native'sspot_price, previouslyRecoverableError). Swap limits convert through a checkedBigUint::from_f64(Bebop, Hashflow and Liquorice cast throughu128).Levels, shared by Bebop, Hashflow, Liquorice and Native; Metric's depth model is its own and is untouched): a negative or non-finite quantity, or a non-positive/non-finite price on a level with quantity, rejects the whole response, and zero-quantity levels are dropped.clientfield now serializes the venue's client (its endpoints — the book ones as well as the quote one — plus timeout and origins; credentials and HTTP pool skipped) instead of the whole feed client;MetricStatehas noclientfield.BebopStateserializesbook(base, quote,last_update_ts, bids/asks as{price, quantity}) where main serialized the prost message with flatf32arrays;NativeState's book nests its ladders and minimums underbids/asks. Recordings made before this branch do not deserialize.MetricMetadata,MetricBidAskResponse,PaginatedMetadataResponse,MetricDepth,MetricDepthBinandq64_to_f64have no public path, and neither does anything innative::models(public on main since feat: integrate native RFQ protocol #1244). Every provider'smodelsmodule is private now.D. Errors
snapshot_feed::errors::FeedError { Connection, Parsing, Fatal, InvalidInput }(from feat(snapshot-feed): add the latest-value snapshot feed layer #1554) is used by feeds, builders and Metric end to end.rfq::errors::RFQErroris crate-private (waspub). It is the quoting path's error, but every function returning it lives in a private module and the states convert it intoSimulationError, so no public signature carried it. Variants unchanged, nowClone. There is no conversion between the two enums.E. Configuration and constants
book::BookFeedConfig { chain, tokens: Arc<HashMap<Bytes, Token>>, min_tvl_usd: f64 }(added in feat(snapshot-feed): add the latest-value snapshot feed layer #1554) becomes required: it is the first argument of every feed builder. A plain struct with public fields, built as a literal, no setters. Tokens carry metadata (the builders previously tookHashSet<Bytes>ids). Quote tokens are not on it and are not defaulted anywhere: passbook::quote_tokens::usd_stablecoins_for_chain(chain)to the four builders that take them.min_tvl_usd(wastvl_thresholdon the builders). Metric's implicit0.0default is gone, and there is no public default constant.rfq::constants::{get_bebop_auth, get_hashflow_auth, get_liquorice_auth, get_native_auth, BebopAuth, HashflowAuth, LiquoriceAuth, NativeAuth},{DEFAULT_METRIC_API_URL, MetricConfig, get_metric_config}, and{BebopOrigins, get_bebop_origins}.rfq::protocols::utils::bytes_to_addressis gone too, replaced by the crate's existingevm::protocol::utils::bytes_to_address. Credentials are constructor arguments; Bebop's origins are builder setters.rfq::constantsis crate-private and no public default constants remain.MetricFeedBuildergainsbid_ask_concurrency(NonZeroUsize)— how many of a poll's per-pool bid/ask requests run at once, default 16 — and itsdefault_feed_config()is the one that is not the shared set: a 20 s cadence, a 15 s request timeout and a 60 s withdrawal age. Metric is the only venue whose poll is many requests (a page of pool metadata, then one per pool), so it costs more per poll and is given more time to finish one; measured live it reprices every 2-3 s against a poll that can take close to 9 s. The two in-repo consumers drop the overrides they used to pass.F. Behaviour
G. Cargo
serde_withis a new optional dep ofbook-feedsandderive_morebecomes an unconditional direct dependency. Both were already in the lock.serde_withalso moves from an inline version in tycho-ethereum to the workspace;tokio-streamis pinned to the workspace version (same 0.1 line, listed for completeness); and tycho-integration-test gainstokio-streamandchrono.tycho_test::execution::encoding::USER_ADDRis nowpub(was a privateconst), so the integration test can identify its simulated flow to venues that screen origins.Behaviour changes worth a heads-up
These compile without a change on the consumer's side and still alter what a running system does:
max_consecutive_failures: None); Bebop used to give up after ten consecutive failed connections. Only a fatal error ends a feed whatever the budget says, and fatal means no attempt can answer differently: an unsupported chain, an unbuildable request. A rejected credential is not one — it is fixed at the venue while the feed keeps trying, and the attempt that follows serves the book again without a restart. Giving up resolves the feed future with the error that ended it, classified as its source classified it, withdraws the published book and drops the sender; it never appears as an item.build(), their endpoints needing a name for the chain that no chain id supplies; Hashflow, Liquorice and Metric resolve its chain id at build and let the venue answer, so the first poll's refusal ends the feed and withdraws its books. Bebop builds for the eight chains its API prices, where main built only for Ethereum and Base. On main Hashflow and Liquorice checked nothing and reached the chain throughChain::id()on every poll and every quote, which panics for a custom chain missing from the registry — so a chain they do not serve was an endless retry loop, and an unregistered one a panicking feed task.build()a configuration it cannot run with. An empty trading key isInvalidInput: Metric answers an empty bearer token with the sameUnknown chainId404 it refuses an unserved chain with, where a key it rejects is a plain 401, so a feed started without one would read its own misconfiguration as the venue's refusal and end. Abase_url()that is not a URL isInvalidInputtoo, where reqwest reports it once per request and the loop retries it for as long as the process runs.rfq:bebopfollowed by the 20 bytes of each token — where main hashed a similar string with keccak256. Same information, no hash, and the protocol system instead of a bare venue name, so a venue that later gets a native integration cannot collide with its book feed. Hashflow's ids carry one more segment, the market maker's name, because a pair's makers are separate books (below). Metric is the exception: its component is a pool, not a pair, and it keeps the pool address. Ids recorded from main do not match any more.get_amount_outfills against the one paying most — but the quote request named nobody, so the venue could answer with a level from another maker, on another maker's depth, at a price the route was never built on. The request carries the maker now and the response is filtered to it. Same defect as Hashflow's above, and still open on main, where the maker filter reached only the all-pairs path feat(simulation): RFQ venues as one component for all pairs with quote reuse rules #1501 added.SimulationError::FatalError, which says there is a bug with this pool or protocol and that it should not be simulated again — a connection that did not hold and a body that did not parse among them. Each lands by what it is now, and three refusals that are none of those are classified where they are read:No maker supports this requestandExceeds supported amounts, Bebop'sInsufficientLiquidityandMinSize. All four are an answer about one trade at one size, which the same book quotes at another. Neither venue publishes a catalogue of its codes, so a code they have not been seen to decline with stays fatal; what is known about each list is recorded beside it.No consumer acts on the difference today: tycho-execution carries it into
EncodingErrorvariant for variant, and both in-repo consumers flatten that into a message. What changes is that a venue's verdict survives to the one deciding what to do about it. In the error counts of a deployment running the old classification, the three refusals above were the largest quoting failures by a wide margin.Noneafter the book is older thanmax_snapshot_age(both transports), or when the feed ends at all — giving up, or the consumer dropping the stream or watch running it — and the next book restores it. On main the last book stayed in place for the length of an outage. Consumers therefore have to handleNoneafter the first snapshot, not only before it, and a consumer that only watches the channel still learns that a feed died.poll_interval(expoll_time) starts from the same 5 s as main for every HTTP feed; a zero interval fails the feed immediately instead of panicking in tokio. It is a floor on the gap between two polls whatever a poll costs: one that outruns the interval is followed by the next as soon as it answers, and the cadence runs on from there — the intervals it ran through are not owed back as a burst of catch-up polls. A consumer that polls a venue more slowly than the 30 s default withdrawal age usually wants to move both knobs together, or every book is withdrawn before the poll that would refresh it; the feed warns rather than refuses, because serving a book only while it is fresh is a legitimate choice when the venue's rate limit fixes the cadence. The integration test pairs its three 30 s pollers with 90 s.backoff_unit; a poller stretchespoll_intervalitself, waiting a multiple of it in place of the next tick, so the schedule ispoll_intervaldoubled up tomax_backoff_exptimes — 5, 10, 20, 40 s on the defaults — and a recovered poll resumes the plain cadence rather than being followed by a catch-up one. On main no HTTP client backed off at all: a provider that was down or rate-limiting was polled every 5 s for the whole outage.[price, quantity]pairs. Zero-quantity levels are dropped at ingestion for every venue (Native's[0, 0]placeholders were the only known producer).MinSize: Request quotes over $1.— and nothing published said so, so a leg worth cents simulated clean and the refusal arrived at encode time, with the quote lost.BebopStatenow holds what one of its quote tokens is worth in USD, which the feed already works out to measure the book's TVL, andget_amount_outreads the trade's worth off the quote side. The floor was measured rather than assumed: bisecting the refusal on Ethereum and Base, in both directions and on three pairs, gives one dollar of the sell side everywhere, and the venue publishes no per-token minimum. In the base-to-quote direction the trade is valued at the maker's own price rather than at mid, so the floor sits half a spread high. Hashflow has held callers to its maker's first level all along; this is Bebop catching up, andBebopState's serialized form gains the price it needs to.min_amount_in()scales the first level's quantity into base units andget_amount_outused to scale the caller's amount back down before comparing, which for roughly 5 % of 18-decimal quantities lands an ulp short and rejected a caller that asked for exactly what it was told to ask for. The floor is compared in base units against the valuemin_amount_in()hands out.get_limitsreports is a complete fill. The limit goes out through atomic units and comes back a hair different — the round trip alone overshoots by up to one ulp — andfillused to leave that hair as an unfilled remainder, soget_amount_outat the advertised limit answered with an error instead of a quote. A remainder within four ulps of the input now counts as spent; a real shortfall is still reported.usd_stablecoins_for_chainreturnsNonefor chains without a curated set; previously an empty set silently filtered out every pair.tracingwith spans. Every feed runs in asnapshot_feedspan carrying the provider, so every event from a source or the loop is attributable; each HTTP fetch runs in apollspan and each decoded WebSocket frame in aws_framespan, whose close timings give the cost of a fetch and of a decode. Binding quote requests run in aquote_requestspan with the pair and amount. Variables are fields, not message text (error,consecutive,component_id,books, …). Level changes: conditions the loop retries on its own (WebSocket read, decode and connect failures) log atwarn, where main logged some of them aterror; per-poll fetch counts and TVL-normalization skips aredebug(Bebop and Native warned per pair per update). New events:infowhen a feed publishes a book while none was servable (first book, and every restoration after a withdrawal) and when a poll or connection succeeds after failures;debugper published snapshot after that, and per decoded frame with the venue's pair count where the source knows it.Consuming the feeds, and what a deployment has to carry
Two things that used to be hidden are in the consumer's hands, and both are visible in the snippet:
Nonein the channel — nothing servable — arrives asBookFeedEvent::Withdrawn, which the consumer has to name rather than skip with acontinue, and which means "stop quoting this provider", not only "nothing yet".Ended(outcome)is where the consumer learns that a provider stopped, and why — it gave up, it ran out, or its task died of a bug. On main the merge task's result was dropped, so a dead feed went unnoticed until someone saw stale prices.In exchange the consumer decides how the feeds are consumed, where the merge layer forced one shape on everyone:
SnapshotFeedStream::spawn(provider, feed)and read it, instead of a merged queue every provider had to share.SnapshotFeedWatch::spawn(provider, feed), keep areceiver()next to the router and readborrow()only when a route is being priced; no stream and no task of your own, and a reader cannot slow the venue's connection.Ended(_), rebuild the feed andaddit again, or drop the provider for the session; on main the retry budget was the library's call.select!can hold the block stream, so books and on-chain state are consumed in one loop.Requesting a binding quote is unchanged in principle and moves to the state:
state.as_indicatively_priced()?.request_signed_quote(params).await. Additions and removals are the consumer's own bookkeeping: compare the snapshot's book ids with the previous view.Deployment
Recorded component ids and states do not carry over. Every pair-keyed id changes (the new scheme is in the behaviour section), and every RFQ and Metric state's serialized form changes with it, so a saved market recording made before this fails to replay against the new states. The protocol-system strings are unchanged, so nothing in a deployment's configuration moves.
Open questions and follow-ups
.../for-solvers/request-for-quote-protocols), stay: the repo has no.gitbook.yamland no redirect map, and other docs deep-link those anchors. Cleaning the URLs wants a redirects entry and is its own change.LiquoriceExecutor__InvalidSelectorreverts under Verification are.quote_timeoutsetters); the price-level stream ontoSnapshotFeed; the five-venue behaviour fixes listed under drift; a shared per-provider quote budget.BookFeedStreams, keeps the ids each venue last published to derive additions and removals, and writes book updates to the base market state instead of relabelling the market with a timestamp.HeaderLike,FeedMessage<H>,StateSyncMessage<H>in tycho-client andTychoStreamDecoder<H>,TryFromWithBlock<T, H>(23 implementations) in tycho-simulation existed to let aTimestampHeaderstand in for a block header on the RFQ path. Removing the parameter is mechanical but breaks the public API of both crates, so it is its own PR. refactor(protocol)!: rename Update.block_number_or_timestamp to block_number #1562 takes the one piece of it that is not: theUpdatefield name.IndicativelyPricednames a pricing quality but is used as a mechanism: "this state needs a round trip to the venue before encoding, and the signed result goes into the calldata". Titan's price-level venues are indicatively priced too and do not implement it, while pre-v1 Metric implemented it to fetch a signed oracle update that was no quote. A name that says what the method does, and a return type not calledSignedQuote, is a tycho-common change touching every RFQ swap encoder, so it is a separate design question.Verification — automated, live per venue, and end to end
Automated.
cargo +nightly fmt --all --checkclean;cargo clippy --workspace --all-targets --all-featuresclean. tycho-simulation--lib: 1793 pass, the 18 failuresRPC_URL-gated EVM tests this branch does not touch (simulation_db, the generic VM hook handler); tycho-integration-test 35 pass; tycho-execution 286 pass with 19RPC_URL-gated failures of the same kind (permit2, the encoding integration tests).Live, per provider. The
rfq/,pamm/andbook/suites run against the real APIs with credentials, network-gated tests included: 321 tests, 320 pass. Bebop's pricing socket and both quote shapes, Hashflow's poll and its binding quote, Liquorice's quote path, Native's orderbook and Metric across nine chains are all exercised. The one failure is Metric on HyperEVM, where the chain's only pool answers"priceProviderStatus": "feed_down"withbidAdj: 0and no depth while an Ethereum pool quotes normally in the same minute — the venue declining to quote, which is what the test asserts against.Live, end to end. One trade quoted on every venue inside the same minute, five times: 5 000 USDC → WETH on Ethereum. The size clears Hashflow's per-pair minimum and sits inside the depth every venue advertises.
Four venues, four wire formats, one ladder implementation, agreeing within 0.054 % on every sample and drifting together as the market moves — the strongest available check that the shared pipeline decodes, orients and fills correctly, since a venue-specific bug in any of those steps shows up as one column diverging.
Metric serves no Ethereum pools. On Base it prices tokenised equities against USDC, where 5 000 USDC buys 21.92 NVDAc against Bebop's 21.87 for the same pair in the same minute — the pAMM and the RFQ families cross-checking each other.
The venue-wide layout was quoted the same way: with
--all-pairs, Hashflow's six per-maker components collapse into one and all four venues still answer the 5 000 USDC trade within 0.01 % of their per-pair prices.Integration test, book feeds against tycho-beta, simulating, encoding and executing per sampled component. Two 900 s runs, one per layout, same flags (
--skip-messages-duration 30), one process at a time:get_amount_outfailuresget_limitsfailuresThe layout does what it is for: 392 components become 19 — one per RFQ venue plus Metric's pools, which stay one per pool — and a route is held to one quote per maker inside each. Fewer components is also why it executes fewer swaps: the harness samples at most
--max-simulationsdirections per component and update, so collapsing a venue's books into one component bounds what one update can simulate.It quotes what it advertises, and every failure in the table has a cause:
get_limitsrefuses a one-sided book: tokenised equities (tGLD, QQQon, CRCLx, INTCon) and SYRUP, which the venue quotes in one direction only.RecoverableError("No liquidity")is the contract this PR gives an empty side, and the harness counts it because it asks every direction.get_amount_outrefused 4 Hashflow swaps on USDC/USDT, each asking one atomic unit more than its book held. The run predates the fix in this PR, which is why the column still shows them. A book of a single level advertises that level as both its minimum and its depth;min_amount_in()rounded it up to a whole unit of the token whileget_limitsrounds the total down, so no size satisfied both and the harness asked the minimum. It rounds down now — a live maker signs one, two, ten and ninety units under its published level, and refuses half of it with code 79 — so the minimum is a size the book can fill.LiquoriceExecutor__InvalidSelector()(0x8c4b1d97), which fix(execution): follow Liquorice to its current settlement contract #1487 fixes by following Liquorice to the settlement contract it deployed on 2026-08-05. The two Bebop ones per run areLimitAmountViolation()from Bebop's own settlement at0xbeb0009a…: the venue enforcing the order's limit amount, which is the guard working.The benchmark numbers in the table at the top are from 2026-09-10 and stand as measured; everything in this section was measured on 2026-10-12.
🤖 Generated with Claude Code