Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more detail to the EKS workaround #15785

Merged
merged 7 commits into from
Oct 11, 2024
Merged

Add more detail to the EKS workaround #15785

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f9770a8
Add more detail to the EKS workaround
bleggett Oct 10, 2024
e3a47bd
Fixup
bleggett Oct 10, 2024
be46ae3
Fixup more
bleggett Oct 10, 2024
aec34b5
Update content/en/docs/ambient/install/platform-prerequisites/index.md
bleggett Oct 11, 2024
050dac6
Update content/en/docs/ambient/install/platform-prerequisites/index.md
bleggett Oct 11, 2024
5825ffa
okay look, strictly speaking based on how the alphabet works the lint…
bleggett Oct 11, 2024
e84f5b5
Update content/en/docs/ambient/install/platform-prerequisites/index.md
bleggett Oct 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .spelling
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1055,6 +1055,8 @@ sds
Searcy
Secura
SecurityCon
SecurityGroupPolicy
SecurityGroups
security_bulletin
selinux
serverless
Expand Down
26 changes: 25 additions & 1 deletion content/en/docs/ambient/install/platform-prerequisites/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,31 @@ spec:

### Amazon Elastic Kubernetes Service (EKS)

If you are using EKS with Amazon's VPC CNI **enabled**, [`POD_SECURITY_GROUP_ENFORCING_MODE` must be explicitly set to `standard`](https://github.com/aws/amazon-vpc-cni-k8s/blob/master/README.md#pod_security_group_enforcing_mode-v1110), or pod health probes (which are by-default silently exempted from all policy enforcement by AWS VPC CNI) will fail. This is because Istio uses a link-local SNAT address for kubelet health probes, which AWS VPC CNI is not aware of, and AWS VPC CNI does not have an option to exempt link-local addresses from policy enforcement.
If you are using EKS
bleggett marked this conversation as resolved.
Show resolved Hide resolved

- with Amazon's VPC CNI
- with Pod ENI trunking enabled
- **and** you are using EKS pod-attached SecurityGroups via [SecurityGroupPolicy](https://aws.github.io/aws-eks-best-practices/networking/sgpp/#enforcing-mode-use-strict-mode-for-isolating-pod-and-node-traffic)

[`POD_SECURITY_GROUP_ENFORCING_MODE` must be explicitly set to `standard`](https://github.com/aws/amazon-vpc-cni-k8s/blob/master/README.md#pod_security_group_enforcing_mode-v1110), or pod health probes (which are by-default silently exempted from all policy enforcement by the VPC CNI) will fail. This is because Istio uses a link-local SNAT address for kubelet health probes, which Amazon's VPC CNI is not aware of, and the VPC CNI does not have an option to exempt link-local addresses from policy enforcement.

You can check if you have pod ENI trunking enabled by running the following command:

{{< text syntax=bash >}}
$ kubectl set env daemonset aws-node -n kube-system --list | grep ENABLE_POD_ENI
{{< /text >}}

You can check if you have any pod-attached security groups in your cluster by running the following command:

{{< text syntax=bash >}}
$ kubectl get securitygrouppolicies.vpcresources.k8s.aws
{{< /text >}}

You can set `POD_SECURITY_GROUP_ENFORCING_MODE=standard` by running the following command, and recycling affected pods:

{{< text syntax=bash >}}
$ kubectl set env daemonset aws-node -n kube-system POD_SECURITY_GROUP_ENFORCING_MODE=standard
{{< /text >}}

### k3d

Expand Down