Update addon-id-troubleshoot.adoc for EKS Pod Identity. #894
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds additional debug step for EKS pod identity implementation. Hey found this out after spending time debugging with 3 different AWS Support engineers. Everything was setup using the AWS Docs from here https://docs.aws.amazon.com/eks/latest/userguide/pod-id-agent-setup.html The same config was working on the support engineer's cluster but failing on our cluster. Then we ran this query on the Cloudwatch Log insights on the eks cluster logs ``` fields @timestamp, @message | filter @logstream like /kube-apiserver/ and @logstream not like /kube-apiserver-audit/ | filter @message like /failed calling webhook/ | sort @timestamp desc ``` Which showed us some errors in the mutating webhooks then we compared the YAML of ours and their pod-identity-webhook MutatingWebhookConfiguration which led us to this discovery. Their webhook was on v1Beta1 while ours was on V1.
|
This pull request is automatically being deployed by Amplify Hosting (learn more). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes:
Adds additional debug step for EKS pod identity implementation.
Hey found this out after spending time debugging with 3 different AWS Support engineers. Everything was setup using the AWS Docs from here https://docs.aws.amazon.com/eks/latest/userguide/pod-id-agent-setup.html . The same config was working on the support engineer's cluster but failing on our cluster. Then we ran this query on the Cloudwatch Log insights on the eks cluster logs
Which showed us some errors in the mutating webhooks then we compared the YAML of ours and their cluster's pod-identity-webhook MutatingWebhookConfiguration which led us to this discovery. Their webhook was on v1Beta1 while ours was on V1.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.