feat(operator): support shmSize on VLLMRuntime for tensor parallelism (#899)

[Anai-Guo]

What

Adds a shmSize field to the VLLMRuntime CRD (deploymentConfig.shmSize). When set, the operator mounts an emptyDir with medium: Memory at /dev/shm, sized to the given quantity.

Fixes #899.

Why

Tensor-parallel vLLM communicates between ranks over shared memory. In containers /dev/shm defaults to a small size (typically 64Mi), which is too small for TP and leads to crashes / Bus error. The Helm chart can be worked around with extra volumes, but the VLLMRuntime operator had no equivalent knob — exactly what the issue asks for (shmSize: 24g).

What changed

• api/v1alpha1/vllmruntime_types.go — new optional ShmSize string on DeploymentConfig.
• internal/controller/vllmruntime_controller.go — when ShmSize != "", append a dshm emptyDir{medium: Memory} volume + a /dev/shm mount on the vLLM container. The size is parsed with resource.ParseQuantity; an unparseable value still yields a Memory-backed /dev/shm (no size limit) rather than panicking the reconcile loop.
• config/crd/bases/...vllmruntimes.yaml — regenerated CRD property.

Behavior

• Field unset → no change (backwards compatible; dshm volume is only added when requested).
• shmSize: "24Gi" → /dev/shm backed by a 24Gi Memory emptyDir.

Example:

spec:
  deploymentConfig:
    shmSize: "24Gi"

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces a new ShmSize configuration option to VLLMRuntime to allow mounting an emptyDir (medium=Memory) at /dev/shm for tensor parallelism. The feedback highlights a bug where updates to shmSize will not trigger a Deployment update because the controller does not compare volumes or volume mounts. Additionally, it is recommended to use *resource.Quantity instead of string to leverage Kubernetes API-level validation and prevent silent failures when parsing invalid quantity strings.

[gemini-code-assist]

Bug: Changes to shmSize will not trigger a Deployment update

The deploymentNeedsUpdate function (lines 951-1108) does not compare the Pod's Volumes or VolumeMounts when determining if the existing Deployment needs to be updated.

As a result, if a user updates the shmSize field on an existing VLLMRuntime custom resource, the operator will detect no changes, and the Deployment will not be updated to mount the /dev/shm volume.

Recommendation

Update deploymentNeedsUpdate to compare the volumes and volume mounts of the expected and actual deployments. For example:

// Compare volumes
if !reflect.DeepEqual(expectedDep.Spec.Template.Spec.Volumes, dep.Spec.Template.Spec.Volumes) {
    log.Info("Volumes mismatch")
    return true
}

// Compare volume mounts for the main container
if !reflect.DeepEqual(expectedDep.Spec.Template.Spec.Containers[0].VolumeMounts, dep.Spec.Template.Spec.Containers[0].VolumeMounts) {
    log.Info("Volume mounts mismatch")
    return true
}

[gemini-code-assist]

Improvement: Silent failure on invalid shmSize quantity

If the user provides an invalid quantity string (e.g., "24Gii"), resource.ParseQuantity will fail, and the error is silently ignored. This results in an emptyDir being mounted with no size limit (which defaults to using the node's memory/tmpfs limits), without any indication to the user or administrator that their configuration was invalid.

Recommendation

Consider logging a warning when parsing fails, or refactoring deploymentForVLLMRuntime to return an error so that the reconciliation loop can fail/requeue and surface the invalid configuration in the resource status.

Alternatively, you can change the type of ShmSize in the API definition to *resource.Quantity (see the other comment) to leverage Kubernetes API-level validation and prevent invalid values from being accepted in the first place.

[gemini-code-assist]

Recommendation: Use *resource.Quantity instead of string

Using a raw string for ShmSize requires manual parsing in the controller and can lead to silent failures or runtime errors if the user inputs an invalid quantity (e.g., "24Gii").

By changing the type to *resource.Quantity (from k8s.io/apimachinery/pkg/api/resource), the Kubernetes API server will automatically validate the field value upon creation or update, rejecting any invalid quantities before they ever reach the operator.

Example

import resource "k8s.io/apimachinery/pkg/api/resource"

// ...

type DeploymentConfig struct {
    // ...
    
    // ShmSize, when set, mounts an emptyDir with medium=Memory at /dev/shm
    // sized to this value (e.g. "24Gi").
    // +optional
    ShmSize *resource.Quantity `json:"shmSize,omitempty"`
}