Skip to content

Sandbox egress: NetworkPolicy union defeats the documented 443-only restriction, and IMDS is reachable from sandbox runs #759

Description

@sabbour

Problem

The sandbox execution deep-dive states that sandbox egress is restricted to HTTPS/443, but that is
not what is enforced on the cluster today. Measured from inside a real Kata sandbox run
(PR #757 capability validation):

  • curl http://169.254.169.254/metadata/instance?api-version=2021-02-01 (Azure IMDS) is
    reachable from the executor sidecar.
  • Plain HTTP (:80) egress to public mirrors succeeds — which is why apt-get works at all.

Root cause

Kubernetes NetworkPolicies are additive. The narrow policy shipped in k8s/base is unioned with
the per-template NetworkPolicy generated by the controller, and the generated policy allows all
ports
to the public CIDR block. The union therefore permits any TCP port to any public address,
including the link-local IMDS address, and the documented "443 only" restriction is not in force.

Impact

  • IMDS reachability is the significant part. The node identity is exposed to any code the model runs
    inside a sandbox unless IMDS itself declines to serve it. This is a confidentiality concern,
    not an isolation break: the sandbox still has no host namespaces, no hostPath, no service-account
    token, and no cross-run workspace access (all separately proven in fix(sandbox): unblock AgentHost warm pool on real AKS Kata with a hardened executor sidecar #757).
  • Documentation currently overstates the enforced control, which is its own problem — reviewers
    reasonably rely on it.

What needs to happen

  1. Decide the intended egress contract. If the sandbox must reach package registries, "443 only" is
    the wrong contract and the docs should say what is actually intended (for example: any port to
    public CIDRs, minus link-local and cloud metadata ranges).
  2. Make the controller-generated per-template policy exclude 169.254.0.0/16 (and the other
    link-local/metadata ranges) from its public-CIDR except list, so the union cannot re-open it.
  3. Consider whether a cluster-wide baseline deny for the metadata range is the more durable control,
    since NetworkPolicy union means any future permissive policy re-opens this.
  4. Correct docs/deep-dive/sandbox-pod-execution.md to describe the enforced behaviour once fixed,
    and add a test/probe assertion so the guarantee is regression-checked rather than asserted.

Evidence

Reproduce with the PR #757 validation tooling:

node scripts/validation/collect-kata-evidence.mjs \
  --image <agent-host image> --name kata-egress --phase capability --cases negative

The negative case prints the IMDS reachability result alongside the other host-access controls.

Notes

Found during the PR #757 Kata capability validation. Pre-existing — not introduced by that PR,
which is why it is tracked separately rather than blocking it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority:p1This sprintsquadSquad triage inbox — Lead will assign to a membersquad:tankAssigned to Tank (Backend Engineer)type:bugSomething broken

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions