Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws eks update-kubeconfig invalid apiVersion #6920

Closed
code-is-art opened this issue May 4, 2022 · 160 comments
Closed

aws eks update-kubeconfig invalid apiVersion #6920

code-is-art opened this issue May 4, 2022 · 160 comments
Assignees
Labels
bug This issue is a bug. eks eks-kubeconfig

Comments

@code-is-art
Copy link

Describe the bug

Update kubectl from v1.23.6 to 1.24.0 and run commands

aws eks update-kubeconfig --name ${EKS_CLUSTER_NAME} --region ${REGION}
kubectl version

I get the following error and exit status 1

error: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1alpha1"

Kubectl must need an updated apiVersion in kubeconfig file. Not sure if this is on aws side or kubectl side.
kubernetes/kubectl#1210

Expected Behavior

No error message when using kubectl and aws eks update-kubeconfig

Current Behavior

error: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1alpha1"

Reproduction Steps

Update kubectl from v1.23.6 to 1.24.0 and run commands

aws eks update-kubeconfig --name ${EKS_CLUSTER_NAME} --region ${REGION}
kubectl version

Possible Solution

No response

Additional Information/Context

No response

CLI version used

whatever is running in aws/codebuild/standard:5.0

Environment details (OS name and version, etc.)

aws/codebuild/standard:5.0

@code-is-art code-is-art added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels May 4, 2022
@tim-finnigan
Copy link
Contributor

Hi @code-is-art thanks for reporting. We just received a feature request related to this: #6916.

There is also a related PR open (#6476) that the team is going to prioritize reviewing.

Are you able to pin to a previous version of Kubernetes as a temporary solution?

@tim-finnigan tim-finnigan self-assigned this May 4, 2022
@tim-finnigan tim-finnigan removed the needs-triage This issue or PR still needs to be triaged. label May 4, 2022
@code-is-art
Copy link
Author

yes v1.23.6 works fine. I modified my pipelines to stay at this version for now. I was doing

curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl

but now I am doing the following as a solution.

curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.23.6/bin/linux/amd64/kubectl

I believe this version of kubectl was released yesterday and my new pipeline picked it up.

@Zheaoli
Copy link

Zheaoli commented May 5, 2022

https://gist.github.com/Zheaoli/335bba0ad0e49a214c61cbaaa1b20306

Here's the workround

@bpourian
Copy link

bpourian commented May 5, 2022

If you are still having issues in your pipeline or after you install make sure to check the version:

curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.23.6/bin/linux/amd64/kubectl mv ./kubectl /usr/local/bin/kubectl

@kyandaks
Copy link

kyandaks commented May 5, 2022

Still experiencing the same issue.
I am using GitHub Runners and the kubectl version on the runners is 1.24.0.
Still figuring out a way to update this to a version less than 1.24.0

@genieai-vikas
Copy link

Facing same issue

@yermukov
Copy link

yermukov commented May 5, 2022

same here

1 similar comment
@beeshalrizal
Copy link

same here

@code-is-art
Copy link
Author

code-is-art commented May 5, 2022

There is a thumbs up button on the first post for the same here crowd. Unless you have something to add to the conversation please use it. No one like to be spammed with same here emails and notifications.

@arb000r
Copy link

arb000r commented May 5, 2022

same here

@TranToan2910
Copy link

TranToan2910 commented May 5, 2022

I have the same problem with GitHub runner.
I switched back to version 1.23.6 using this action and it works now
https://github.com/Azure/setup-kubectl

      - uses: azure/[email protected]
        with:
          version: 'v1.23.6'
        id: install

@filipepratalima
Copy link

filipepratalima commented May 5, 2022

Even simply getting the kubectl version is showing this error:

`➜ ~ sudo apt search kubectl
Sorting... Done
Full Text Search... Done
kubectl/kubernetes-xenial,now 1.24.0-00 arm64 [installed]
Kubernetes Command Line Tool

kubetail/focal 1.6.5-2 all
Aggregate logs from multiple Kubernetes pods into one stream

➜ ~ kubectl version
error: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1alpha1"`

@David-Tamrazov
Copy link

David-Tamrazov commented May 5, 2022

I have the same problem with GitHub runner.
I switched back to version 1.23.6 using this action and it works now
https://github.com/Azure/setup-kubectl

worked like a charm for us; thank you @TranToan2910 ! ❤️

@lcube45
Copy link

lcube45 commented May 5, 2022

  • task: KubectlInstaller@0
    displayName: 'Install Kubectl'
    inputs:
    kubectlVersion: '1.23.6'

@FernandoAlencarJr
Copy link

same problem when using github actions ...

@tim-finnigan
Copy link
Contributor

Thanks all for your feedback. As someone mentioned earlier, please 👍 the original post if you are affected by this and only leave a comment if you have additional information to share.

Kubernetes v1.24.0 was released Tuesday, May 3rd according to here: https://kubernetes.io/releases/

Several users have reported this breaking their workflows, many of which involve the use of GitHub Actions/Runners. The team is actively looking into this further and I will share an update as soon as I have more information.

In the meantime, reverting to v1.23.6 is a temporary solution as mentioned in comments here and here.

m25n added a commit to wavefrontHQ/wavefront-collector-for-kubernetes that referenced this issue May 5, 2022
the aws-cli produces a kubeconfig that is incompatible with v1.24.x version of kubectl.
see aws/aws-cli#6920 for more details
@jcs47
Copy link

jcs47 commented May 5, 2022

I have the same problem with GitHub runner.
I switched back to version 1.23.6 using this action and it works now
https://github.com/Azure/setup-kubectl

I am having issues with that action:

Error: Error: Kubectl '1.23.6' for 'amd64' arch not found.

m25n added a commit to wavefrontHQ/helm that referenced this issue May 5, 2022
    the aws-cli produces a kubeconfig that is incompatible with v1.24.x version of kubectl.
    see aws/aws-cli#6920 for more details
m25n added a commit to wavefrontHQ/helm that referenced this issue May 5, 2022
    the aws-cli produces a kubeconfig that is incompatible with v1.24.x version of kubectl.
    see aws/aws-cli#6920 for more details
m25n added a commit to wavefrontHQ/helm that referenced this issue May 5, 2022
* the aws-cli produces a kubeconfig that is incompatible with v1.24.x version of kubectl.
    see aws/aws-cli#6920 for more details
* Turn EKS integration tests back on
@lasanthak
Copy link

For MAC:

First verify the version:
$ kubectl version --client=true

Download and install 1.23.6:
https://dl.k8s.io/release/v1.23.6/bin/darwin/amd64/kubectl

Fixed!

@johnzheng1975
Copy link

Upgrade aws client version!!!!!!
This works!!!!!

@jvsell
Copy link

jvsell commented Mar 16, 2023

Consegui resolver da seguinte forma:

  1. Atualizei para o AWS CLI versão 2.11.3 que é a ultima disponível neste momento.
    https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#getting-started-install-instructions

  2. Rodei novamente o update do kubeconfig.
    aws eks update-kubeconfig --name CLUSTER_NAME --region REGION

E funcionou.

@Kejzin
Copy link

Kejzin commented Mar 21, 2023

I have the samme issue:
error: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1alpha1"

I'm using nevest aws version

aws-cli/2.11.4 Python/3.11.2 Linux/5.15.0-67-generic exe/x86_64.ubuntu.22 prompt/off

And I'm using nevest to date kubectl
Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.3", GitCommit:"9e644106593f3f4aa98f8a84b23db5fa378900bd", GitTreeState:"clean", BuildDate:"2023-03-15T13:40:17Z", GoVersion:"go1.19.7", Compiler:"gc", Platform:"linux/amd64"} Kustomize Version: v4.5.7

when i switched v1alpha1 to v1beta1 it worked but is it a good workaround?

@rahulbaid
Copy link

For MacOS users still facing this issue,

I upgraded the aws CLI version (from 2.4.6 to 2.11.4). If this brew upgrade awscli does not work, try this:

aws --version
rm '/usr/local/bin/aws'
brew install awscli
brew link --overwrite awscli
aws --version
aws configure sso
aws sso login
kubectl config delete-context ${CLUSTER_NAME}

and now copy-paste the AWS environment variables (short-term credentials) into the terminal session

aws sts get-caller-identity
aws eks update-kubeconfig --name ${CLUSTER_NAME} --region ${REGION_NAME}
kubectl version

@Vadim-Zenin
Copy link

Check kubectl config file:
kubectl config view | grep 'client.authentication.k8s.io'
You could see
apiVersion: client.authentication.k8s.io/v1alpha1

Copy current kubectl config

if [[ -f ${HOME}/.kube/config ]]; then
    printf "INFO: copy ${HOME}/.kube/config file\n"
    cp ${HOME}/.kube/config ${HOME}/.kube/config.$(date +%Y%m%d-%H%M)
fi

Reconfigure kubectl

AWS_DEFAULT_REGION="${AWS_DEFAULT_REGION:-eu-west-1}"
AWS_PROFILE="${AWS_PROFILE:-<replace_me>}"
EKS_NAME="<replace_me>"
aws eks list-clusters --profile ${AWS_PROFILE} --region ${AWS_DEFAULT_REGION}
aws eks update-kubeconfig --name ${EKS_NAME} --profile ${AWS_PROFILE} --region ${AWS_DEFAULT_REGION}
aws eks describe-cluster --name ${EKS_NAME} --query cluster.status --profile ${AWS_PROFILE} --region ${AWS_DEFAULT_REGION}

Check if the issue exists:
kubectl version
Client Version: version.Info{Major:"1", Minor:"23+", GitVersion:"v1.23.17-eks-a59e1f0", GitCommit:"567b6a1a716729b7a8c4196cf3b61d3c84df88f3", GitTreeState:"clean", BuildDate:"2023-03-09T19:54:56Z", GoVersion:"go1.19.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.10-eks-48e63af", GitCommit:"9176fb99b52f8d5ff73d67fea27f3a638f679f8a", GitTreeState:"clean", BuildDate:"2023-01-24T19:17:48Z", GoVersion:"go1.19.5", Compiler:"gc", Platform:"linux/amd64"}
aws --version
aws-cli/2.11.11 Python/3.11.2 Linux/5.4.0-137-generic exe/x86_64.ubuntu.20 prompt/off

@NBrendel
Copy link

Got it working on Ubuntu 22.04 by downgrading the kubectl version to 1.23.17 (installed via snap)

sudo snap install kubectl --channel=1.23/stable --classic

@mballoni
Copy link

Same issue here. The min working version was 1.23 as well.

@vbarba88
Copy link

Worked by upgrading the aws cli version: from 2.2.41 to 2.11.9

@ikersuen
Copy link

Same issue still exist when using

kubectl client version: v1.24.x
kubectl server version; v1.24.x

Still requires to downgrade to use client version v1.23.x for resolving the problem.

@arhimondr
Copy link

Thanks @rahulbaid , worked for me

@ratkokorlevski
Copy link

Worked by upgrading the aws cli version: from 2.2.41 to 2.11.9

@rmsys
Copy link

rmsys commented Jul 13, 2023

Thanks @rahulbaid! Worked for me, executing the second step session.

Ubuntu 20.04.6 LTS
aws-cli/2.13.0
kubectl v1.27.3

@charlesxjq
Copy link

Try to update the kubeconfig with changing "apiVersion: client.authentication.k8s.io/v1alpha1" into "apiVersion: client.authentication.k8s.io/v1beta1" and it works for me. I think maybe it causes by the upgrade of the kubernetes apiVersion.

@OmpahDev
Copy link

Having the issue with latest aws-cli and latest kubectl.

@huitingyeong
Copy link

same issue. This worked for me:

  • Updating AWSCLI from 2.4.9 to 2.13.19
  • Run aws eks update-kubeconfig --region XXX --name XXX (this essentially updates ~/.kube/config and changes apiVersion to "client.authentication.k8s.io/v1beta1"

@pallavikamboj123
Copy link

same issue. This worked for me:

  • Updating AWSCLI from 2.4.9 to 2.13.19
  • Run aws eks update-kubeconfig --region XXX --name XXX (this essentially updates ~/.kube/config and changes apiVersion to "client.authentication.k8s.io/v1beta1"

Thankyou so much. This worked for me.

nickumia-reisys added a commit to GSA-TTS/datagov-brokerpak-eks that referenced this issue Oct 5, 2023
@lccoronel
Copy link

same issue. This worked for me:

  • Updating AWSCLI from 2.4.9 to 2.13.19
  • Run aws eks update-kubeconfig --region XXX --name XXX (this essentially updates ~/.kube/config and changes apiVersion to "client.authentication.k8s.io/v1beta1"

This worked for me.

ben851 pushed a commit to cds-snc/notification-manifests that referenced this issue Dec 18, 2023
@ulidtko
Copy link

ulidtko commented Feb 2, 2024

But what about awscli v1 ?

Since #4947 is still unresolved, many environments cannot install, much less update, awscli v2. Because pip install "awscli >= 2" does not work.

No EKS for those, right? 😉

@ulidtko
Copy link

ulidtko commented Feb 2, 2024

With:

  • awscli v1 — aws-cli/1.22.34 Python/3.10.12 Linux/5.15.0-91-generic botocore/1.27.67
  • EKS cluster version 21 — cluster.platformVersion: eks.21 (in aws eks describe-cluster)
  • v1alpha1 get-token apiVersion — client.authentication.k8s.io/v1alpha1

— I got it working by downgrading kubectl to latest available version before 1.24.0 (kubectl dropped v1alpha1 in 1.24.0).

I.e. in Ubuntu, in my case, the latest version before 1.24.0 was 1.23.17:

sudo apt install kubectl=1.23.17-00
sudo apt-mark hold kubectl

hold disables further auth-updates.

@anhnguyenviet109
Copy link

same issue. This worked for me:

* Updating AWSCLI from 2.4.9 to 2.13.19

* Run aws eks update-kubeconfig --region XXX --name XXX (this essentially updates ~/.kube/config and changes apiVersion to "client.authentication.k8s.io/v1beta1"

This works for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. eks eks-kubeconfig
Projects
None yet
Development

No branches or pull requests