fix(costs): version default-section storage removal accounting

[QuantumExplorer]

Summary

Fixes #683. Combining a BasicStorageRemoval with a SectionedStorageRemoval can discard the default identifier's entire epoch map, losing both its existing bytes and the basic bytes being added. Preserve that section under GROVE_V4 while retaining exact GROVE_V1–V3 cost results.

Three operator arms had the defect: Basic + Sectioned, Sectioned + Basic, and Basic += Sectioned. Sectioned += Basic was already correct and remains unchanged on every version.

Version selection and scope

• Add grovedb_versions.storage_costs.add_basic_storage_removal_to_sectioned_storage_removal, set to 0 for V1–V3 and 1 for V4.
• Carry the version through the existing cost operators with a thread-local selector and scoped guard. The selector defaults to legacy behavior. GroveDB sectional-delete and batch entry points, and the innermost Merk apply entry point, select the requested version for synchronous execution and storage-cost accumulation.
• Keep the guard !Send and !Sync, so its destructor restores the originating thread's selector. Mark it must_use and document synchronous scope use, reverse-order drops for nested guards, and the prohibition on holding it across .await. The closure helper does not extend its scope to futures returned by the closure.

Aggregation outside a GroveDB call, including Drive's combine_cost_operations, still uses legacy arithmetic unless the consumer installs the guard using the applicable GroveVersion's storage-cost slot. That downstream integration remains a separate change; upgrading GroveDB alone does not correct every external cost sum.

Production impact

Drive's storage-flags callback returns basic/basic removals for unflagged elements and sectioned/sectioned removals for flagged elements, with no-removal results where appropriate for zero bytes. Mixed basic/sectioned arithmetic arises when those results are aggregated. The GroveDB deletion tests deliberately use custom basic/sectioned callbacks to exercise the affected arithmetic.

Only the default identifier's section is dropped. Drive separates that section into FeeResult.removed_bytes_from_system before calculating identity refunds; identity-owned sections survive in both legacy and corrected arithmetic. The 0-of-155 and 0-of-302 test results below demonstrate missing removal accounting, not missing identity refunds on each Drive deletion. See Platform's fee conversion. Preserving V1–V3 outputs maintains historical cost compatibility without asserting a demonstrated identity-refund or consensus defect.

Validation

• Exact-map regressions cover all four mixed operator arms with the default section present with UNKNOWN_EPOCH, present without it, and absent. They preserve other owners and epochs, pin legacy losses, and assert that Sectioned += Basic is identical unguarded, under version 0, and under version 1.
• Version tests pin the selector to 0/0/0/1 for V1–V4.
• A custom-callback single deletion reports exactly 155 removed bytes under V4 and an empty map under V3. A two-item batch reports exactly 302 bytes with both epochs preserved under V4 and an empty map under V3.
• Compile-fail regressions reject moving the guard between threads or requiring it to be Sync. Runtime tests check thread isolation, nesting, restoration to legacy, and restoration after a nested panic.

Local checks:

cargo test --locked --offline -p grovedb-costs -p grovedb-version
cargo test --locked --offline -p grovedb --all-features --lib basic_plus_default_sectioned_removal_cost
cargo test --locked --offline -p grovedb --all-features --lib batch_delete_
cargo clippy --locked --offline --workspace --all-features -- -D warnings
cargo fmt --all --check

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 923a2754-4fa2-419f-8f59-4a4edba13dad

📥 Commits

Reviewing files that changed from the base of the PR and between 0e904fe and 6fbb628.

📒 Files selected for processing (12)

• costs/src/storage_cost/removal.rs
• costs/tests/coverage_regression.rs
• grovedb-version/src/tests.rs
• grovedb-version/src/version/grovedb_versions.rs
• grovedb-version/src/version/v1.rs
• grovedb-version/src/version/v2.rs
• grovedb-version/src/version/v3.rs
• grovedb-version/src/version/v4.rs
• grovedb/src/batch/mod.rs
• grovedb/src/batch/single_deletion_cost_tests.rs
• grovedb/src/operations/delete/mod.rs
• merk/src/merk/apply.rs

🚧 Files skipped from review as they are similar to previous changes (8)

• grovedb-version/src/version/v1.rs
• grovedb-version/src/tests.rs
• grovedb-version/src/version/v2.rs
• costs/src/storage_cost/removal.rs
• grovedb-version/src/version/grovedb_versions.rs
• grovedb/src/batch/mod.rs
• costs/tests/coverage_regression.rs
• grovedb/src/operations/delete/mod.rs

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

This PR fixes storage-removal arithmetic that dropped default-section entries. It adds version-gated behavior, keeps legacy behavior for V1–V3, enables corrected behavior in V4, and applies the setting across batch, deletion, and Merk operations.

Changes

Storage Removal Addition Versioning

Layer / File(s)	Summary
Versioned storage removal arithmetic costs/src/storage_cost/removal.rs	Adds scoped version selection. Corrected arithmetic preserves the default section. Legacy arithmetic remains available.
Storage-cost version configuration grovedb-version/src/version/grovedb_versions.rs, grovedb-version/src/version/v1.rs, grovedb-version/src/version/v2.rs, grovedb-version/src/version/v3.rs, grovedb-version/src/version/v4.rs, grovedb-version/src/tests.rs	Adds storage-cost version settings. V1–V3 use version 0. V4 uses version 1. Tests verify these values.
Runtime guard integration grovedb/src/batch/mod.rs, grovedb/src/operations/delete/mod.rs, merk/src/merk/apply.rs	Applies the configured storage-removal version during batch application, deletion, and tree application.
Removal arithmetic and deletion-cost validation costs/tests/coverage_regression.rs, grovedb/src/batch/single_deletion_cost_tests.rs	Tests addition and assignment combinations, default-section preservation, corrected V4 behavior, and legacy V3 behavior.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 6fbb6

The change preserves legacy accounting for older versions while correcting default-section removal accounting in V4; no actionable merge-blocking risk remains beyond normal checks and review.

Sequence Diagram(s)

sequenceDiagram
  participant GroveVersion
  participant GroveDBOperation
  participant Merk
  participant RemovalArithmetic
  GroveVersion->>GroveDBOperation: provide storage-cost version
  GroveDBOperation->>RemovalArithmetic: activate scoped version guard
  GroveVersion->>Merk: provide storage-cost version
  Merk->>RemovalArithmetic: apply version during walker and commit
  RemovalArithmetic-->>GroveDBOperation: calculate versioned removal total
  RemovalArithmetic-->>Merk: calculate versioned removal total

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The changes preserve default-section removals for the affected addition paths and add regression tests for the required combinations in issue #683.
Out of Scope Changes check	✅ Passed	The versioning, integration updates, and regression tests directly support the storage-removal accounting fix and its compatibility requirements.
Docstring Coverage	✅ Passed	Docstring coverage is 82.61% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 23 functions across 12 files.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the main change: versioned accounting for default-section storage removal costs.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/fix-683-storage-removal-default-section

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.60%. Comparing base (6ec8fee) to head (f94265e).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #726      +/-   ##
===========================================
+ Coverage    92.56%   92.60%   +0.04%     
===========================================
  Files          307      307              
  Lines        95705    95756      +51     
===========================================
+ Hits         88585    88672      +87     
+ Misses        7120     7084      -36     

Components	Coverage Δ
grovedb-core	90.65% <100.00%> (+0.04%)	⬆️
merk	93.71% <100.00%> (+<0.01%)	⬆️
storage	91.70% <ø> (ø)
commitment-tree	96.38% <ø> (ø)
mmr	95.11% <ø> (ø)
bulk-append-tree	92.78% <ø> (ø)
element	98.05% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[QuantumExplorer]

This is Claude (AI-assisted). Pushed a rework (commit 5d827ed) addressing the consensus-safety problem in the previous version.

The bug being fixed: the default-section-drop only ever affected three of the four basic-into-sectioned arms — Add: Basic+Sectioned, Add: Sectioned+Basic, and AddAssign: Basic+=Sectioned. The fourth, AddAssign: Sectioned+=Basic, already reinserted the default section correctly in every shipped version.

The prior version routed all four arms through the version selector, so the legacy/v0 path dropped the default section for the fourth arm too — i.e. it changed shipped v1/v2 output (its own regression test even asserted the wrong 0 for that case). Since storage-removal bytes feed fee/state accounting, that's a v1/v2 replay divergence.

Change:

• Sectioned += Basic now calls the default-section-preserving helper unconditionally (it was never broken, so it must not be gated). The three genuinely-buggy arms remain gated to v3.
• Fixed the regression test to assert the preserved total (13) on the legacy path, with a comment explaining why that arm is version-independent.
• Documented why the thread-local selector exists (the version-sensitive combination happens inside Add/AddAssign, reached through ~hundreds of version-less StorageCost/OperationCost aggregation sites; the costs crate has no grovedb-version dep) and that its default (0/legacy) is the safe one.

On the thread-local: I looked at threading the version explicitly instead, but the combination flows through StorageCost/OperationCost operator overloads at ~951 version-less call sites, so explicit threading would require a large, risky refactor of the cost system. The thread-local is the pragmatic localization; it's now documented and the legacy default is safe.

Verified: grovedb-costs + grovedb-version tests pass (incl. v3_uses_fixed_basic_to_sectioned_storage_removal_addition and the corrected regression test); grovedb deletion-cost tests pass (incl. latest_delete_preserves_basic_plus_default_sectioned_removal_cost); clippy clean across all three.

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[QuantumExplorer]

This is Claude (AI-assisted). Assessment of how needed this fix is, and why the PR is being marked deferred rather than merged now.

TL;DR: the fix is correct, V4-gated, CI-green and the legacy path is pinned byte-exact — but the bug has no effect on any fee, refund, pool credit, or root hash Platform computes today. It is hygiene / future-proofing, not a live defect, so it is parked rather than carrying a thread-local version selector into grovedb-costs for a no-impact bug.

What the bug can actually lose

• Only the default identifier ([0; 32]) section is dropped by the three buggy arms (Add: Basic + Sectioned, Add: Sectioned + Basic, AddAssign: Basic += Sectioned). Identity-owned sections (owner_id ≠ default) are never touched, so identity fee refunds are unaffected in every shipped version.
• What is lost is "system" bytes: the Basic removal being folded in, plus the default section it was folded into.

How Drive reaches the buggy arms

• Within a single Drive op: never. StorageFlags::split_removal_bytes (grovedb-epoch-based-storage-flags/src/split_removal_bytes.rs) returns Basic/Basic for unflagged elements and Sectioned/Sectioned for flagged ones, so a deletion's key + value removals always combine through a same-kind (correct) arm.
• Across ops in one batch / cost accumulation: only when an unflagged removal (Basic) is aggregated with a no-owner-flagged removal (default-identifier section), and only in the Basic += Sectioned order (Sectioned += Basic was always correct). The only production no-owner flags in rs-drive are the identity-tree elements (add_new_identity: "no reason to store the owner id"); documents, contracts, etc. all carry an owner. Identities are never deleted and their values rarely shrink, so the trigger is rare.

What happens when it does fire

• Drive routes both BasicStorageRemoval amounts and the default-identifier section to FeeResult.removed_bytes_from_system (rs-drive/src/fees/op.rs, the BasicStorageRemoval(amount) => (FeeRefunds::default(), amount) / remove(&Identifier::default()) arms).
• Nothing in rs-drive-abci consumes that field: it is summed in FeeResult::checked_add_assign, zeroed in estimates, and not part of validate_fees_of_event (which compares total_base_fee / storage_fee / processing_fee). Storage fees are added_bytes × credit_per_byte; refunds come only from owner sections.
• So even when triggered: no fee, refund, pool credit, or committed state changes.

Why defer rather than merge

• The Codex "medium" rating describes the arithmetic in isolation; the Drive-level impact is nil today.
• The fix is cheap and safe, but its design cost is real: a thread-local Cell<u16> selector in grovedb-costs installed via RAII guards at the Merk/GroveDB entry points (the operator impls can't carry a grove_version and grovedb-costs has no grovedb-version dependency). It also leaves a gap — removal aggregation outside a GroveDB call (Drive's combine_cost_operations, FeeResult sums) stays on the legacy arithmetic under V4 unless Drive installs the guard itself.
• Both the mechanism and the gap only matter if something starts consuming removed_bytes_from_system (e.g. system storage accounting). Until then there is nothing to fix in production.

State of the branch (so it can be picked back up quickly)

• Rebased on develop, V4-gated (v1..v3 = 0, v4 = 1), CI green, CodeRabbit clean.
• Regressions pin the exact shipped figures: under GROVE_V3 the combined removal is SectionedStorageRemoval({}) → 0 of 155 added bytes; under GROVE_V4 {default: {UNKNOWN_EPOCH: 155}} → 155.
• Re-open / merge when: Platform starts consuming removed_bytes_from_system, or a GroveDB consumer needs Basic + default-section removals to aggregate correctly, or the next time a V4 cost-gate sweep is being merged anyway (zero marginal risk then). Otherwise this can be closed with an entry in docs/audit-non-issues.md ("default-section loss only affects removed_bytes_from_system, which has no consumer").

[QuantumExplorer]

This is Claude (AI-assisted). Brought the branch up to date and validated it against the September 2026 audit addendum on #683 (group C008). Draft status and the deferred label are untouched — whether to lift them stays with the maintainer.

Rebase. Rebased the four existing commits onto develop at 6ec8feec (31 commits behind). It applied cleanly: the storage_costs slot in GroveDBVersions and its v1..v4 initialisers slotted in next to the newer V4 gates without conflicts. No production code changed in this push; the fix itself is as described in the PR body. CI green on the new head (c00356e5): all three test shards, linting, formatting, security audit, codecov patch/project. CodeRabbit skips reviews on drafts and there are no open inline comments.

Validation matrix added (the addendum's three items):

1. Pin legacy and corrected mixed-order / default-epoch totals. costs/tests/coverage_regression.rs now pins the exact resulting map for all four arms — Basic + Sectioned, Sectioned + Basic, Basic += Sectioned, Sectioned += Basic — over three default-section shapes (present with UNKNOWN_EPOCH, present without it, absent), under version 0 (V1..V3) and 1 (V4). The input default owner already carries {3: 20, 5: 30, UNKNOWN: 7} next to an identity-owned {2: 40}, with 11 incoming basic bytes, so the legacy result {identity: {2: 40}} (40 of 108) shows both losses the audit names: the owner's existing epoch attribution and the incoming basic bytes. The no-UNKNOWN_EPOCH shape pins the legacy result at an empty map (0 of 31) — that is the branch codecov flagged as uncovered before.
2. Sectioned += Basic unchanged on legacy. Asserted byte-identical unguarded, under version 0, and under version 1 for all three shapes; it produces the corrected map in every case.
3. Entry-point guards and external aggregation. Every in-repo caller of the mixed arms (KeyValueStorageCost::combined_removed_bytes at the storage-batch commit, plus StorageCost/OperationCost +=) runs inside the guarded scopes; delete_up_tree_while_empty_with_sectional_storage funnels into the guarded batch apply. New GroveDB test: two epoch-flagged items deleted in one apply_batch_with_element_flags_update batch — V4 yields {default: {3: v₃, 5: v₅, UNKNOWN: k₃ + k₅}}, 302 of 302 added bytes; V3 pinned to the empty legacy map, 0 of 302. The guard's default-to-legacy and nest/restore behaviour is pinned too. The requirement that aggregation outside a GroveDB call (Drive's cost sums) install the guard to get the corrected arithmetic is now documented in docs/crates/costs.md as well as in the PR body.

Local runs: cargo test -p grovedb-costs, -p grovedb-version, -p grovedb-merk, -p grovedb --all-features (3182 passed; the full,estimated_costs subset does not compile on current develop without unsafe-dump-load, unrelated to this PR), cargo fmt --all, cargo clippy --workspace --all-features -- -D warnings.

Why it was deferred (unchanged). The three buggy arms only drop the default-identifier ([0; 32]) section; identity-owned sections, and therefore identity fee refunds, are never touched in any shipped version. Drive reaches the buggy arms only when an unflagged Basic removal aggregates with a no-owner-flagged removal, and both amounts land in FeeResult.removed_bytes_from_system, which nothing in Platform consumes for fees, refunds, pool credits, or state. So the defect changes no consensus-relevant figure today; the fix is hygiene that carries a thread-local version selector into grovedb-costs and leaves the Drive-side aggregation gap noted above. The PR is now rebased, fully pinned and CI-green, so it can be merged at zero marginal cost alongside the next V4 cost-gate sweep, or when Platform starts consuming removed_bytes_from_system; otherwise it can be closed with an entry in docs/audit-non-issues.md.