PrivateDocumentStore: append-only element type for fixed-size opaque entries with committed config and provable range reads

[QuantumExplorer]

Motivation

Platform is adding private document types: a data contract can declare a document type private — no public index structure; each document is stored as a hiding commitment (32B) plus a ciphertext of uniform, contract-declared size; recipients discover documents by scanning and trial-decrypting rather than querying indices. Uniqueness is enforced by a sibling tag set (existing ProvableCountTree, nullifier-style), and the state transitions are funded from the shielded pool — all Platform-side concerns.

Phase one covers non-modifiable private documents only: entries are write-once, nothing is ever updated, deleted, or proven-against later. That means the store needs no Sinsemilla frontier and no anchor — it is a pure append-only log with provable reads. (Modifiable private documents are a later phase built on the anchored DataCommitmentTree of #783 plus a nullifier set; explicitly out of scope here.)

This issue adds the element type for that store. Behaviorally it is fully generic — an append-only store of fixed-size opaque entries; GroveDB never interprets a "document" — the name simply describes its Platform use.

Design

• ElementType::PrivateDocumentStore = 15 with counted semantics, plus NonCountedPrivateDocumentStore = 143 following the existing +128 convention (12–14 and 140–142 are taken by Mmr/BulkAppend/DenseAppendOnlyFixedSize trees; 15/143 are the next free pair). Matching TreeType variant carrying the chunk power.
• Thin wrapper over BulkAppendTree — the same relationship CommitmentTree has to it, minus the Sinsemilla frontier.
• Committed config: the element value carries {entry_size, chunk_power}, and the config is bound into the state root, e.g. blake3("pds_state" || config_hash || bulk_state_root) — so the declared entry size is consensus-visible and a proof can never be reinterpreted under a different config. Precomputed empty-root constant with a runtime-equivalence test (mirroring EMPTY_COMMITMENT_TREE_STATE_ROOT).
• Insert validates exact entry size; there is no per-entry delete or update — immutability is enforced by the type, matching what the Platform grammar's default promises.
• Reads: get-by-position, count (from the counted variant), and paginated position-range reads with proofs — the scanning hot path (every client walks "all entries since my cursor"). Range reads should be implemented at the BulkAppendTree layer so the anchored DataCommitmentTree (DataCommitmentTree: generic commitment tree for data (configurable hash domain, entry format, payload size) #783), which embeds a BulkAppendTree, inherits them for free.
• Fail closed: new discriminants rejected outright by older versions; everything gated under the next GROVE_V* flag.

Work plan

• 1. Element + TreeType plumbing. New discriminants 15/143 in grovedb-element/src/element_type.rs, TreeType variant in merk/src/tree_type/mod.rs, config carrier + serialization in grovedb-element/src/element/{constructor,helpers,visualize}.rs.
• 2. State-root binding. pds_state composite root with config hash; empty-root constant + equivalence test.
• 3. Operations. private_document_store_insert (size-validated append), get_value, count in a new grovedb/src/operations/private_document_store.rs, sharing the BulkAppendTree storage adapter.
• 4. Range reads with proofs at the BulkAppendTree layer: paginated position-range fetch, chunk-aligned, with proof generation/verification in operations/proof/{generate,verify}.rs; absence falls out of the provable count (position ≥ count).
• 5. Match-site sweep. is_tree(), TryFrom<u8>, display names, merk/src/element/{costs,delete,get,reconstruct}.rs, replace_subtree_root, is_empty_tree, batch policy (batch/mod.rs, batch_structure.rs). Mechanical but consensus-critical.
• 6. verify_grovedb integrity walk (bulk-tree ↔ config consistency, entry-size check).
• 7. Costs. Entry-size-parametrized storage costs; batch/estimated_costs/{average_case,worst_case}_costs.rs, merk/src/tree_type/costs.rs.
• 8. Versioning. GROVE_V* gating on discriminants and ops; rejection tests against the previous version.
• 9. Benches + tests. Append/scan benches; proof coverage tests.
• 10. grovedbg/debugger rendering (non-consensus, lower priority).

Note: grovedb/src/replication has no support for commitment-tree-family types today; PrivateDocumentStore inherits that state-sync gap and should be included when it is addressed (see the workstream-D2 note in #783).

Non-goals

• Sinsemilla frontier, anchors, membership witnesses — that is DataCommitmentTree (DataCommitmentTree: generic commitment tree for data (configurable hash domain, entry format, payload size) #783).
• Nullifier sets and modifiable documents (later phase, Platform-composed).
• The uniqueness tag set (existing ProvableCountTree).
• Platform-side contract grammar, validation, encryption, or fee logic.

[QuantumExplorer]

This is Claude, closing on behalf of the maintainer.

The last open item, state sync, landed in #840 (6ec8fee): a populated PrivateDocumentStore replays through PrivateDocumentStore::append, so the committed entry_size from the target's hash-verified element is enforced on every wire entry, and finalize recomputes and checks the config-bound state root (compute_non_merk_state_root gained the PDS arm, including the config-parametrized empty root). The rest of this issue (crate, operations, V4 gating, proofs, verify_grovedb walk) was already in place per the August triage.

Tests: state_sync_private_document_store_round_trip, state_sync_private_document_store_tampered_pages_rejected.