Skip to content

fix(nvca): apply primary-PV selector in model cache cleanup - #441

Merged
balajinvda merged 4 commits into
NVIDIA:mainfrom
mesutoezdil:mesutoezdil/fix/modelcache-pv-selector
Jul 29, 2026
Merged

balajinvda merged 4 commits into
NVIDIA:mainfrom
mesutoezdil:mesutoezdil/fix/modelcache-pv-selector

Conversation

@mesutoezdil

@mesutoezdil mesutoezdil commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

TL;DR

The model cache cleanup selector was left empty because the result of
labels.Selector.Add was discarded. An empty selector matches every object, so
idle cleanup listed every PV in the cluster instead of only model-cache primary
PVs. This skewed the model cache backend count metric and made the cleanup loop
scan all PVs each tick. Reassign the Add result so the requirement applies, plus
a regression test.

Issues

Closes #440

Testing

go test ./pkg/storage/... with envtest passes. Added TestPrimaryPVSelector,
which fails before the fix (empty selector matches an empty label set) and passes
after. No QA needed.

Checklist

  • I am familiar with the Contributing Guidelines.
  • I have signed off my commits for Developer Certificate of Origin (DCO) compliance.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Summary by CodeRabbit

  • Bug Fixes

    • Corrected persistent volume label selector construction so model cache cleanup only targets volumes with the required primary-volume label.
    • Prevents unlabeled volumes from being matched incorrectly during cleanup.
  • Tests

    • Added coverage validating selector behavior for empty/unlabeled volumes and for volumes carrying the expected primary-volume label value.

@mesutoezdil
mesutoezdil requested a review from a team as a code owner July 25, 2026 09:53
@mesutoezdil
mesutoezdil requested a review from balajinvda July 25, 2026 09:53
@coderabbitai

coderabbitai Bot commented Jul 25, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The primary PV selector now retains its label requirement, and a unit test verifies that it rejects unlabeled objects and matches PVs with the primary label.

Changes

Primary PV selector

Layer / File(s) Summary
Selector construction and matching validation
src/compute-plane-services/nvca/pkg/storage/modelcache_cleanup.go, src/compute-plane-services/nvca/pkg/storage/modelcache_cleanup_test.go
primaryPVSel is reassigned after adding the primary PV label requirement, with tests covering matching and non-matching label sets.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Suggested reviewers: balajinvda

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is valid Conventional Commits and accurately describes the bug fix in model cache cleanup.
Linked Issues check ✅ Passed The selector fix and regression test satisfy the linked issue's requirements.
Out of Scope Changes check ✅ Passed The PR only changes the cleanup fix and its test, with no unrelated scope visible.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"


Comment @coderabbitai help to get the list of available commands.

@mesutoezdil
mesutoezdil force-pushed the mesutoezdil/fix/modelcache-pv-selector branch from bec4165 to 2647a51 Compare July 25, 2026 12:23
labels.Selector.Add returns a new selector and does not mutate the
receiver, so the discarded return left primaryPVSel empty. An empty
selector matches every object, so the idle model cache cleanup listed
every PV in the cluster instead of only model-cache primary PVs. This
skewed the model cache backend count metric and made the cleanup loop
scan all PVs each tick.

Reassign the Add result so the primary-PV requirement is applied, and
add a regression test on the selector.

Closes NVIDIA#440

Signed-off-by: mesutoezdil <mesudozdil@gmail.com>
@mesutoezdil
mesutoezdil force-pushed the mesutoezdil/fix/modelcache-pv-selector branch from 2647a51 to 569d778 Compare July 27, 2026 07:38

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/compute-plane-services/nvca/pkg/storage/modelcache_cleanup_test.go (1)

38-46: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use a table-driven test for the selector cases.

This test covers multiple scenarios inline. Convert them to table-driven cases so additional selector behaviors can be added consistently.

As per coding guidelines, tests covering multiple scenarios should use table-driven tests.

Proposed refactor
 func TestPrimaryPVSelector(t *testing.T) {
-	assert.False(t, primaryPVSel.Matches(labels.Set{}),
-		"empty label set must not match; selector should require the primary-PV label")
-	assert.True(t, primaryPVSel.Matches(labels.Set{primaryPVLabelKey: "true"}),
-		"a PV carrying the primary-PV label must match")
+	tests := []struct {
+		name     string
+		labelSet labels.Set
+		want     bool
+	}{
+		{
+			name: "empty labels do not match",
+			labelSet: labels.Set{},
+			want: false,
+		},
+		{
+			name:     "primary PV label matches",
+			labelSet: labels.Set{primaryPVLabelKey: "true"},
+			want:     true,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			assert.Equal(t, tt.want, primaryPVSel.Matches(tt.labelSet))
+		})
+	}
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/compute-plane-services/nvca/pkg/storage/modelcache_cleanup_test.go`
around lines 38 - 46, Convert TestPrimaryPVSelector into a table-driven test
with named cases for the empty label set and a set containing primaryPVLabelKey.
Iterate over the cases with subtests, asserting each expected Matches result
while preserving the existing failure messages or equivalent scenario-specific
diagnostics.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/compute-plane-services/nvca/pkg/storage/modelcache_cleanup_test.go`:
- Around line 38-46: Convert TestPrimaryPVSelector into a table-driven test with
named cases for the empty label set and a set containing primaryPVLabelKey.
Iterate over the cases with subtests, asserting each expected Matches result
while preserving the existing failure messages or equivalent scenario-specific
diagnostics.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: acd7b72b-cd5e-49c7-9244-0ad32ef9b4d7

📥 Commits

Reviewing files that changed from the base of the PR and between 2647a51 and 569d778.

📒 Files selected for processing (2)
  • src/compute-plane-services/nvca/pkg/storage/modelcache_cleanup.go
  • src/compute-plane-services/nvca/pkg/storage/modelcache_cleanup_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/compute-plane-services/nvca/pkg/storage/modelcache_cleanup.go

@balajinvda balajinvda left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good.

@balajinvda
balajinvda enabled auto-merge July 29, 2026 04:26
@balajinvda
balajinvda added this pull request to the merge queue Jul 29, 2026
Merged via the queue into NVIDIA:main with commit 2c3661b Jul 29, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nvca model cache cleanup lists every PV due to discarded selector Add() result

2 participants