Skip to content

feat(dapi): composite document queries on the getDocuments V1 wire - #4599

Merged
QuantumExplorer merged 1 commit into
feat/composite-document-queries-drivefrom
feat/composite-document-queries-wire
Sep 5, 2026
Merged

feat(dapi): composite document queries on the getDocuments V1 wire#4599
QuantumExplorer merged 1 commit into
feat/composite-document-queries-drivefrom
feat/composite-document-queries-wire

Conversation

@QuantumExplorer

Copy link
Copy Markdown
Member

Issue being fixed or feature implemented

Second PR of the composite-query stack (core in #4598): puts composite document queries on the typed getDocuments V1 wire and serves them from drive-abci. A composite query is a page plus sub-queries derived from its proven results (joins, lookups, counts, siblings), answered as one merged proof; the motivating case is a social feed whose per-page enrichment is today a burst of dependent DAPI calls.

What was done?

Wire (platform.proto)

  • GetDocumentsRequestV1.sub_queries (field 14, repeated SubQuery): each sub-query carries its own contract (empty = the page's), document type, fixed where_clauses / order_by, optional limit, a Kind (DOCUMENTS | COUNT) and an optional Binding { source, source_property, field } whose source is 0 for the page or n for sub_queries[n-1]. Presence selects composite mode; the request's own clauses describe the page. chained and sub_queries are mutually exclusive. No CBOR anywhere: typed clauses only, like the rest of V1.
  • ResultData.composite (variant 7, CompositeDocuments): the page documents plus one SubQueryResult per sub-query (Documents or CountEntries) on the no-proof path. The proof path is the single merged proof in the standard envelope.
  • build.rs: serde default on sub_queries so mock vectors captured before the field stay wire-compatible (same rule as chained).

drive-abci (dispatch/composite.rs)

Routes before the SELECT machinery. Gates: page limit required in [1, max_query_limit]; selects empty or a single DOCUMENTS projection; group_by, having, cursors, offset and time-range clauses (page or sub-query) refused; each distinct sub-query contract fetched once; kinds, limits and bindings decoded into drive's typed shapes; drive's validate failures surface as query errors. Proof path: query_composite_documents_with_proof under the standard proof envelope. No-proof path: documents serialized with their own type and contract, counts as CountEntry rows.

An old node ignores field 14 (proto3 unknown field) and serves the plain page; the verifier in #4598 refuses that page-only proof whenever a sub-query derived anything.

Mechanical: every existing GetDocumentsRequestV1 literal gains sub_queries: Vec::new(); the proof verifier's result-variant name table gains the composite arm.

How Has This Been Tested?

dispatch/composite.rs tests against the yappr-feed fixture plus the dashpay contract:

  • should_return_the_page_and_every_sub_result_without_proof: like counts, quoted-post join, cross-contract profile lookup (a missing profile is a proven absence)
  • should_prove_end_to_end_through_the_v1_wire: the wire proof verifies through DriveCompositeDocumentQuery::verify_composite_documents_proof rebuilt client-side
  • should_require_an_explicit_page_limit, should_reject_chained_and_composite_together, should_reject_an_unknown_sub_query_contract, should_surface_shape_rejections_as_query_errors

The whole document_query::v1 module (103 tests, chained included) passes; clippy --all-targets -D warnings clean on drive-abci, dapi-grpc, drive-proof-verifier and dash-platform-queries; dash-sdk and wasm-sdk compile against the new proto.

Regenerated JS gRPC clients follow in a separate commit on this branch (Docker regen).

Breaking Changes

None. Additive wire fields; old clients and old nodes are unaffected (an old node answering a composite request fails closed on the client).

Checklist

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: df283f2f-42ee-4b3d-a78a-b739f9089ac0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@thepastaclaw

thepastaclaw commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

🕓 Ready for review — 55 ahead in queue (commit 5d3b151)
Queue position: 56/59 · 2 reviews active
ETA: start ~2026-09-06 10:13 UTC · complete ~2026-09-06 11:08 UTC (median 54m across 30 recent reviews; 2 slots)
Queued 5m ago · Last checked: 2026-09-05 09:30 UTC

`GetDocumentsRequestV1.sub_queries` (field 14): each entry is a typed
sub-query (contract, document type, fixed where/order clauses, limit,
DOCUMENTS or COUNT) with an optional binding `<field> IN <values>` whose
values the node derives from the page's, or an earlier sub-query's,
proven documents. Presence selects composite mode: the request's own
clauses describe the page; `chained` and `sub_queries` are mutually
exclusive. `ResultData.composite` (variant 7) carries the page and one
result per sub-query (documents, or count entries keyed by the bound
value) for the no-proof path; the proof path is the single merged proof
in the standard envelope, verified client-side by re-deriving the whole
composition from the proven page.

drive-abci intercepts composite requests before select routing
(`dispatch/composite.rs`): the page limit is required (1..=max), every
SQL-shaped knob, cursor and time-range clause is refused, each distinct
sub-query contract is fetched once, and drive's shape validation
surfaces as client-attributable query errors. Old nodes ignore the
field and serve a page-only proof, which the verifier refuses.

Existing V1 request literals gain the new field; the serde default
keeps mock vectors captured before it wire-compatible.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@QuantumExplorer
QuantumExplorer force-pushed the feat/composite-document-queries-drive branch from ec02c48 to 34a30e4 Compare September 5, 2026 09:23
@QuantumExplorer
QuantumExplorer force-pushed the feat/composite-document-queries-wire branch from 70e5ef4 to 5d3b151 Compare September 5, 2026 09:23
@QuantumExplorer
QuantumExplorer merged commit 5d3b151 into feat/composite-document-queries-drive Sep 5, 2026
3 of 4 checks passed
@QuantumExplorer

Copy link
Copy Markdown
Member Author

Closed by GitHub as merged when a mistaken force-push briefly collapsed the stack's branches onto one commit; nothing from this PR is in v4.2-dev. Replacement with the same branch and content: #4603.

QuantumExplorer pushed a commit to QuantumExplorer/yappr that referenced this pull request Sep 8, 2026
One composite `getDocuments` request now answers a For You page and
everything a card needs to render it, under a single merged proof: the
posts, their like/repost/reply/quote counts, the posts they quote, the
authors' profiles and DPNS names, and (logged in) the viewer's own
likes, reposts and bookmarks on the page. The SDK derives every
sub-query from the proven page, so the responding node cannot steer
any of it. About ten round trips per page become one.

- lib/feed/composite-feed-page.ts builds the query from the contract
  topology, decodes the result into the page plus a
  `PreloadedEnrichment`, seeds the DPNS and profile caches, attaches
  quoted posts, and reports `null` when the surface is unavailable
  (an evo-sdk without `documents.composite`, a pre-v6 contract, or a
  recent failure with a one-minute backoff), so the legacy loaders
  keep working unchanged.
- lib/feed/load-for-you-feed.ts tries the composite page first and
  translates the feed's id cursors into the range clause the
  composite surface paginates with.
- useProgressiveEnrichment accepts the preloaded slices, merges them
  at once and only queries what they did not cover; PostCard and the
  per-card fallbacks are untouched.
- dpnsService.seedUsernames and
  unifiedProfileService.seedProfileDocuments let proven lookups warm
  the batch resolvers (with a short negative cache for proven
  absences).

Needs an evo-sdk release carrying the composite surface
(dashpay/platform#4601 and its stack) and nodes serving it
(dashpay/platform#4599); until then the capability check keeps this
change inert.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@thephez thephez added the dapi-endpoint DAPI endpoint addition or modification label Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dapi-endpoint DAPI endpoint addition or modification

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants