feat(cloud): metadata coercion, batched get_texts, list_documents, and erase on store delete#97
Merged
Davidobot merged 11 commits intoJul 21, 2026
Conversation
… delete
Three migration-parity gaps between the cloud handle and the local one:
- add/add_vectors now stringify int/float/bool metadata values through the
local handle's own _coerce_metadata before the wire (the server contract
is strict str->str; local-ergonomics code must not 422 on {"year": 2020}).
- get_texts rides the by-id browse (ids=..., include_text=True): one
request per hundred ids instead of one per id, filtered to the requested
ids so a plain-page answer from an older server can only under-answer.
- delete_store gains erase=True — the control plane's data-subject erasure
(no grace window, restore refuses, next sweep hard-deletes).
Codex review: routing every get_texts batch through browse regressed least-privilege credentials — browse is search-scoped, so a key holding only read:text failed a previously valid call. The batched by-id browse stays the fast path; a 403 falls back to the per-id text endpoint (the pre-batching shape), which is exactly what read:text grants.
Codex review round 2: filtering strays out of a plain-page answer (an
older server ignoring the by-id fields) could silently omit requested
documents that exist — get_texts([existing_id]) answering {} where the
per-id endpoint answered text. A stray document now sends the whole
request through the text endpoint, and any id absent from its page is
confirmed per-id — which also preserves the old path's answers for
TTL-hidden documents. On a current control plane the extra cost is one
request per genuinely absent id only.
…ation verb
Browse-backed, local-record-shaped ({id, metadata, chunk_count}), same
filter grammar and after/limit keyset cursor as the local handle, walking
the whole match set in 100-document pages when unpaged. Keyword-only
max_documents/timeout bounds fail the walk closed before another page is
fetched, for callers enumerating stores that may have outgrown them.
idas0
changed the title
The previous prose commit swapped em dashes for colons, which keeps the same rhythm. Rewrite those sentences properly (splits, parenthetical folds, sparing semicolons) and remove internal phase markers from the module docstring and section banners.
CloudError now carries the response's Retry-After and the read retry sleeps that instead of a fixed 250ms, capped by the remaining visibility budget. list_documents checks its deadline after each page fetch too, so a slow final page raises TimeoutError instead of returning past the bound.
The list_documents deadline now clamps each page's read-your-writes retry window, and a 425 that outlives the budget surfaces as the walk's TimeoutError instead of a CloudError after the handle's full 30s visibility wait. A created-but-never-written store (the 'nothing has been pushed' 404) reads as empty like an unprovisioned one; a held min_seq still answers 425, so a session's own writes cannot be hidden.
…rsist The pre-worker-join teardown hook closed every open handle, so a process exit persisted a writable handle's un-persisted in-memory state. A serving tier applying provisional records over a pushed snapshot must never publish them: discard_at_exit=True routes the hook to discard() instead. An explicit close() still persists.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Parity gaps between the cloud handle and the local one: places where code written against a local
lodedb.LodeDBbehaves differently (or worse) throughlodedb.cloud. The erase flag needs a control plane that knows it; the response'spurge_aftersays which contract applied.What changed
add/add_many/add_vectors/add_vectors_manynow stringify int/float/bool metadata values through the local handle's own_coerce_metadatabefore the wire. The server contract stays strict str-to-str; code written against the localdb.addergonomics ({"year": 2020}) no longer 422s, and refused value types get the same error as locally. Absent metadata staysNoneon the wire.get_textsbatches, exact semantics preserved: by-id browse pages of 100 do the bulk work (one request per hundred ids instead of one per id). Anything a page cannot answer authoritatively falls back to the single-id text endpoint: an id absent from its page (genuinely gone, TTL-hidden, or an older control plane that ignored the by-id fields), a stray document nobody asked for (definitely such a control plane), or a 403 (a least-privilegeread:text-only key; browse is search-scoped). On a current control plane the extra cost is one request per genuinely absent id only.list_documentsonCloudStore: the local handle's enumeration verb over browse. Local-record-shaped rows ({id, metadata, chunk_count}; the localcontent_hashis an on-disk artifact detail with no serving-tier counterpart), the same filter grammar assearch, and the sameafter/limitkeyset cursor. An unpaged call walks the whole match set in 100-document pages; keyword-onlymax_documents(ValueError) andtimeout(TimeoutError) bound that walk fail-closed, the timeout checked around every page fetch so a slow final page raises instead of returning past the budget. Like every read it honors the session's read-your-writes floor.delete_store(..., erase=True): the control plane's data-subject erasure (no grace window, restore refuses, next sweep hard-deletes), on bothCloudClientand the tenancy-boundClient.Retry-Afterwhen it sends one (250ms fallback, capped by the handle's visibility budget).CloudErrorcarries the parsed header asretry_after.Review
Ran
codex review --base origin/mainuntil clean: round 1 flagged the scope regression forread:text-only keys (fixed with the 403 fallback), round 2 flagged silent partial results against older control planes (fixed with the stray-detection + per-id confirmation above), round 3 found nothing actionable. A later cross-repo compatibility review (passport + OreCloud + this SDK) flagged the fixed-pace 425 retry and the pre-fetch-only timeout check; both fixed here.Testing
tests/test_cloud_store_verbs.py(19 passed): thelist_documentsset (record shape + full-walk paging, filter/cursor forwarding, both fail-closed bounds, the late-final-page timeout, unprovisioned-store empty answer), the coercion set (values, refusal, None-stays-None),get_textsbatching + per-id confirmation, plain-page distrust, the text-only-key fallback, the?erase=truequery string (a dropped flag would silently downgrade an erasure to a grace delete), and the retry pacing set (server ask honored, 250ms fallback, budget cap,Retry-Afterparsing onCloudError). Surface-parity, routing-hint, and client-api suites green.ruff checkclean.