Repository navigation
fix: preserve explicit projected volumes; strip only kubelet auto-mount - #5
Merged
Merged
Conversation
StripOrphanedVolumes was dropping every projected volume from the
source deployment spec, but the original intent (per its comment) was
to strip the kubelet-injected default ServiceAccount token mount —
which has a `kube-api-access-<random>` name per pod and would collide
if it ended up duplicated in the bridge pod's spec.
The blanket strip also took out legitimately-declared projected
volumes that the kubelet does NOT auto-add back:
- `azure-identity-token` (audience: `api://AzureADTokenExchange`)
used for AKS-style workload identity → Cosmos DB / Azure AD auth
- explicit `aws-iam-token` declarations
- any other projected ConfigMap / Secret / DownwardAPI volumes a
service intentionally declares
For AWS IRSA the EKS Pod Identity webhook silently re-injects the
volume at admission, so AWS-backed services worked anyway. Azure has
no such webhook in venus — the volume must be in the spec or the
projected token file never materializes. Services using
@vercel/cosmosdb hit:
AuthenticationRequiredError: ENOENT: no such file or directory,
open '/var/run/secrets/azure/tokens/token'
and every Cosmos call 500s. Bridge then propagates a stripped pod
spec into the devcontainer's `copyFiles` list — because the Azure
mount isn't on the pod at all, it's not in the list and not copied
locally either.
Narrow the filter to match only `kube-api-access-*` volume names so
explicit projections survive. The kubelet still auto-mounts a fresh
`kube-api-access-<random>` at pod-create time, so we keep the
original defense without taking out everything else.
# Test
Added unit tests covering:
- explicit `azure-identity-token` / `aws-iam-token` projections
survive; their mounts survive
- `kube-api-access-*` volumes are still stripped; their mounts are
still stripped
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
StripOrphanedVolumeswas dropping every projected volume from the source deployment spec. The original intent (per its doc comment) was to strip the kubelet-injected default ServiceAccount token mount —kube-api-access-<random>— whose name has a per-pod random suffix that would collide if a source spec already declared one with that exact prefix.The blanket strip also took out legitimately-declared projected volumes that the kubelet does not auto-add back:
azure-identity-token(audience:api://AzureADTokenExchange) used for AKS workload identity → Cosmos DB / Azure AD auth.aws-iam-tokendeclarations.For AWS IRSA the EKS Pod Identity webhook silently re-injects the projected
aws-iam-tokenvolume at admission, so AWS-backed services worked by accident. Azure has no such webhook in our staging cluster — the projected volume must be in the spec or the token file never materializes.Symptom for any
@vercel/cosmosdb-using service running through bridge:…and every Cosmos call 500s. Because the projected mount also isn't in the live pod's
VolumeMountsafterStripOrphanedVolumesruns, it's missing frompodVolumeMountPaths, so it's also missing from the devcontainer feature'scopyFileslist and never gets pulled into the local container either.Fix
Narrow the filter from "all projected volumes" to "volumes named
kube-api-access-*". The kubelet still auto-mounts a freshkube-api-access-<random>at pod-create time regardless of what's in the spec, so the original defense is unchanged. Everything else (explicit AKS / IRSA / custom projections) survives intact.Test plan
Added
transforms_test.gocovering:azure-identity-token/aws-iam-tokenprojections survive; their mounts on the container also survive.kube-api-access-*volumes (and their mounts) are still stripped.Validation against a real service
Reproduces the bug on
eddie/devbox-project-image-fallbackinvercel/api:bridge create -s /tmp/rendered/manifests.yaml api-devbox→ bridge pod has noazure-identity-tokenvolume → POST/v1/devbox→ 500 with theENOENTabove.bridge remove api-devbox && bridge create -s …): bridge pod gets the projected volume → live pod has the mount →copyFilesincludes/var/run/secrets/azure/tokens→ local devcontainer has the federated token at the expected path → MSAL exchanges via STS → Cosmos auth succeeds.I'll attach a second-pass log once I've rebuilt + re-curl'd to confirm.
Follow-ups (separate, not in this PR)
copyFilesis a one-shot at devcontainer-create time. If a bridge session outlives the token TTL, MSAL will fail on refresh. Worth either documenting the limitation or wiring a refresh loop into the devcontainer feature.