feat: Multiple pools per guard - #22
Conversation
f5519bc to
7e9fc77
Compare
BREAKING CHANGE: local DRAM residencies and transfer results carry ordered pool-and-slot locations. Signed-off-by: Kapil Arya <kapila@nvidia.com>
BREAKING CHANGE: Guard claims and grants describe one jointly owned group of pool regions. Signed-off-by: Kapil Arya <kapila@nvidia.com>
7e9fc77 to
2a0fa68
Compare
Signed-off-by: Kapil Arya <kapila@nvidia.com>
BREAKING CHANGE: capacity_needed_callback now receives an ordered list of pool name and slot count pairs. Signed-off-by: Kapil Arya <kapila@nvidia.com>
There was a problem hiding this comment.
🟡 Changes recommended
It introduces an API-breaking bug in KVCRCore.fetch() defaulting expected_layout to [""] (failing for non-empty single-pool configs) and has at least one recovery decoding error-path that can escape as an uncaught msgspec.ValidationError.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR extends KVCR’s Guard/service and recovery pipeline from a single-pool model to supporting multiple pools per Guard-owned allocation, including multi-descriptor block layouts, pool-aware recovery wire format, and updated capacity-pressure semantics.
Changes:
- Introduces ordered pool descriptors on the Guard protocol wire and binds recovery handback terms to ordered pool geometry/identity.
- Updates local DRAM residency/eviction to manage grouped allocations across multiple pools and multi-descriptor layouts per key.
- Refactors and expands unit/integration tests and documentation to match the new pool-group semantics.
File summaries
| File | Description |
|---|---|
| tests/unit/test_recovery_mirror.py | Updates recovery wire round-trip tests for pool-aware G2 locations and mirror construction. |
| tests/unit/test_recovery_journal.py | Extends snapshot/terms tests to include ordered pool descriptors and allocation identity. |
| tests/unit/test_kvcr.py | Adjusts core/guard startup tests and residency assertions for multi-pool and G3 constraints. |
| tests/unit/test_kvcr_service.py | Reworks service/registry claim tests to validate grouped allocations and pool descriptors. |
| tests/unit/test_kvcr_service_workflow.py | Updates end-to-end daemon workflow tests for pool naming and page-aligned sizing. |
| tests/unit/test_kvcr_remote_source.py | Updates remote FW DRAM wire shape for multi-descriptor destinations and better malformed logging checks. |
| tests/unit/test_kvcr_local_dram.py | Adds multi-descriptor and multi-pool residency/eviction/capacity-pressure behavior tests. |
| tests/unit/test_guard.py | Refactors Guard tests to use new configure/handback APIs and pool-group behavior. |
| tests/unit/test_guard_protocol.py | Adds pool descriptor wire-contract validation and claim/grant layout checks. |
| tests/unit/test_guard_integration.py | Adds integration coverage for two-pool group failover and reclaim behavior. |
| tests/unit/test_g3.py | Adds regression test around G3/full behavior with local slot reuse semantics. |
| tests/unit/_kvcr_test_utils.py | Updates test helpers for nested descriptor lists and capacity callback plumbing. |
| src/kvcr/remote_fw_dram.py | Changes start_write payload decoding and internal ops to support per-key descriptor lists. |
| src/kvcr/recovery_journal.py | Makes recovery wire G2 carry pool-name+slot pairs; binds handback terms to pool descriptors. |
| src/kvcr/policy_runtime.py | Extends eviction queue to track weights (slot counts) and total reclaimable weight. |
| src/kvcr/local_dram.py | Implements multi-pool local DRAM management, multi-descriptor layouts, and per-pool capacity pressure. |
| src/kvcr/local_disk.py | Extracts G3 slot-geometry validation and adapts store path to list-based sources. |
| src/kvcr/kvcr_service.py | Updates service/registry semantics and grant shape for grouped pool allocations. |
| src/kvcr/guard.py | Refactors Guard recovery state to configure pool descriptors atomically and hand back without layout args. |
| src/kvcr/guard_protocol.py | Adds _PoolDescriptor to the wire and validates grant pool layout/offset/size contract. |
| src/kvcr/core.py | Enables multi-pool layouts, multi-descriptor normalization, and per-pool capacity pressure signaling. |
| src/kvcr/api.py | Updates capacity callback type annotation to list of (pool_name, num_slots). |
| docs/dev-guide.md | Updates service daemon documentation for Guard-owned pool groups and ordered pool layout terms. |
| docs/design_overview.md | Updates API contract docs for per-pool capacity pressure requests and multi-pool layouts. |
Review details
- Files reviewed: 24/24 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Restore the base constructor spelling to keep the multi-pool diff focused. Signed-off-by: Kapil Arya <kapila@nvidia.com>
Restore base snapshot-term ordering and journal formatting; retain malformed-frame coverage while sharing repeated setup. Signed-off-by: Kapil Arya <kapila@nvidia.com>
Cover native Guard transfers and reclaim with multiple descriptors. Consolidate overlapping transfer, malformed-input, and capacity cases. Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapila@nvidia.com>
while keys may span a variable ordered set of pool-qualified slots