|
| 1 | +# Amazon ECR "Public" credentials helper script for Kubernetes |
| 2 | + |
| 3 | +Amazon ECR "Public" credentials helper script for Kubernetes (`ecr-public-creds-helper-for-k8s` for short) allows your Kubernetes clusters pull public container images from [Amazon ECR Public](https://aws.amazon.com/blogs/aws/amazon-ecr-public-a-new-public-container-registry/) registries **as authenticated users** to get the limit upgraded to `10` pulls per second which is `1` for unauthenticated users as described [here](https://docs.aws.amazon.com/AmazonECR/latest/public/public-service-quotas.html), and unlimited data bandwidth as described [here](https://aws.amazon.com/ecr/pricing/). |
| 4 | + |
| 5 | +`ecr-public-creds-helper-for-k8s` is one of the workarounds to access ECR Public as authenticated users from your Kubernetes clusters until 1) Amazon ECR Public get supported by the upstream Kubernetes project and/or 2) Official Amazon ECR Public support for AWS Fargate by Amazon EKS. |
| 6 | + |
| 7 | +`ecr-public-creds-helper-for-k8s` runs in your cluster as a Kubernetes CronJob every 8 hours by default. It authenticates against ECR Public and stores the auth token as Kubernetes Secrets within namespaces you specified. |
| 8 | + |
| 9 | +Each pod (even on AWS Fargate) will reference that Kubernetes Secret in its namespace by specifying the `imagePullSecrets` field in the PodSpec. You may also want to patch the `default` service account in each namespace to avoid writing `imagePullSecrets` in all PodSpecs, see the comments at the [entrypoint.sh#L21](entrypoint.sh#L21) for further details. |
| 10 | + |
| 11 | +See the "[Create a Secret by providing credentials on the command line](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#create-a-secret-by-providing-credentials-on-the-command-line)" section and the "[Create a Pod that uses your Secret](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#create-a-pod-that-uses-your-secret)" in the Kubernetes documentation to understand how it works. |
| 12 | + |
| 13 | +## Motivations |
| 14 | + |
| 15 | +This project aims to fill the gap between Amazon EKS (Kubernetes, both EC2 and Fargate) and Amazon ECR Public. There are two pain points for users at the time of release of this repository as follows: |
| 16 | + |
| 17 | +1. Kubernetes supports Amazon ECR (the private one) to pull private ECR images with automatic auth via IAM roles, but still the latest version (1.20, at this moment) of Kubernetes doesn't support ECR Public yet. Thus there is no straightforward way to pull ECR Public container images from EKS/Kubernetes clusters as authenticated users. This means users are forced to access ECR Public as unauthenticated users from their Kubernetes clusters, resulting that pulls from ECR Public could be throttled easily and frequently. |
| 18 | +1. [A PR in the "awslabs/amazon-ecr-credential-helper" GitHub repository](https://github.com/awslabs/amazon-ecr-credential-helper/pull/253) could solve the pain point #1 someday, but the "awslabs/amazon-ecr-credential-helper" itself cannot be used for EKS/Fargate workloads by its design. EKS/Fargate users, not only EKS/EC2 users, obviously need a way to use ECR Public as authenticated users, so this is also the pain point that this project addresses. |
| 19 | + |
| 20 | +## Installation |
| 21 | + |
| 22 | +### Step 0 - Clone repo |
| 23 | + |
| 24 | +```shell |
| 25 | +$ git clone https://github.com/aws-containers/amazon-ecr-public-creds-helper-for-k8s.git |
| 26 | +$ cd amazon-ecr-public-creds-helper-for-k8s |
| 27 | +``` |
| 28 | + |
| 29 | +### Step 1 - Build and Push creds-helper container image |
| 30 | + |
| 31 | +```shell |
| 32 | +$ export CREDS_HELPER_CONTAINER_IMAGE=<your-creds-helper-container-image-name-here> |
| 33 | + |
| 34 | +$ docker build -t ${CREDS_HELPER_CONTAINER_IMAGE} . |
| 35 | + |
| 36 | +$ docker push ${CREDS_HELPER_CONTAINER_IMAGE} |
| 37 | +``` |
| 38 | + |
| 39 | +### Step 2 - Create namespace |
| 40 | + |
| 41 | +Create a namespace for `ecr-public-creds-helper-for-k8s` to run as a CronJob in your Kubernetes cluster. |
| 42 | + |
| 43 | +```shell |
| 44 | +$ kubectl apply -f namespace.yaml |
| 45 | +namespace/ecr-public-creds-helper created |
| 46 | +``` |
| 47 | + |
| 48 | +### Step 3 - Create service account |
| 49 | + |
| 50 | +Create a service account to allow `ecr-public-creds-helper-for-k8s` to edit Kubernetes secrets. |
| 51 | + |
| 52 | +```shell |
| 53 | +$ kubectl apply -f serviceaccount.yaml |
| 54 | +serviceaccount/sa-secrets-editor created |
| 55 | +clusterrole.rbac.authorization.k8s.io/secrets-editor created |
| 56 | +clusterrolebinding.rbac.authorization.k8s.io/edit-secrets created |
| 57 | +``` |
| 58 | + |
| 59 | +### Step 4 - Create IAM role |
| 60 | + |
| 61 | +Create an AWS IAM role to allow `ecr-public-creds-helper-for-k8s` to authenticate against Amazon ECR Public. We use the mechanism called [IAM Roles for Service Accounts (IRSA)](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) to map it to the service account which you created in the previous step. |
| 62 | + |
| 63 | +If you have not enabled IRSA in your Kubernetes cluster yet, please follow the [IRSA documentation](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) and/or the [blog post](https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/) for enabling IRSA for your Kubernetes cluster. |
| 64 | + |
| 65 | +#### Steps with eksctl |
| 66 | + |
| 67 | +We're going to use `eksctl` here to show the steps to create and map the IAM role in an EKS cluster, but you can also use the AWS CLI, the AWS management console, CloudFormation, Terraform or whatever you want to use. |
| 68 | + |
| 69 | +```shell |
| 70 | +$ export POLICY_ARN=$(aws iam create-policy --policy-name AmazonECRPublicAuthOnlyPolicy --policy-document file://iam-permission.json --query Policy.Arn --output text) |
| 71 | +## Check you've created the policy successfully |
| 72 | +$ echo ${POLICY_ARN} |
| 73 | +arn:aws:iam::YOUR_AWS_ACCOUNT_ID:policy/AmazonECRPublicAuthOnlyPolicy |
| 74 | + |
| 75 | +$ export EKS_CLUSTER_NAME=<your-eks-cluster-name-here> |
| 76 | + |
| 77 | +$ eksctl create iamserviceaccount --cluster=${EKS_CLUSTER_NAME} \ |
| 78 | + --name=sa-secrets-editor \ |
| 79 | + --namespace=ecr-public-creds-helper \ |
| 80 | + --attach-policy-arn=${POLICY_ARN} \ |
| 81 | + --override-existing-serviceaccounts \ |
| 82 | + --approve |
| 83 | +``` |
| 84 | + |
| 85 | +### Step 5 - Configure and Apply |
| 86 | + |
| 87 | +#### Configure cronjob.yaml |
| 88 | + |
| 89 | +Edit the [cronjob.yaml](cronjob.yaml) before running `ecr-public-creds-helper-for-k8s` in your Kubernetes cluster. |
| 90 | + |
| 91 | +```shell |
| 92 | +$ vim cronjob.yaml |
| 93 | +``` |
| 94 | + |
| 95 | +There are two **required** fields to change. |
| 96 | + |
| 97 | +##### 1. `image` field |
| 98 | + |
| 99 | +Replace `${CREDS_HELPER_CONTAINER_IMAGE}` in [line.22 in the cronjob.yaml](cronjob.yaml#L22) with your creds helper container image name which you built and pushed in the Step 1. |
| 100 | + |
| 101 | +##### 2. `env.value` field |
| 102 | + |
| 103 | +Replace the value (`default foo bar`) of `TARGET_NAMESPACES` environment variable in [line.26 in the cronjob.yaml](cronjob.yaml#L26) with a space-delimited list which includes one or multiple Kuberentes namespaces where your pods need the auth token for ECR Public. |
| 104 | + |
| 105 | +Let's say you want to pull ECR Public container images in three namespaces (`default`, `prometheus`, `my-app`) with auth token, then the `env.value` field will look like: `value: "default prometheus my-app"`. |
| 106 | + |
| 107 | +`ecr-public-creds-helper-for-k8s` will store the auth token as Kubernetes Secrets in these namespaces. |
| 108 | + |
| 109 | +#### Apply cronjob.yaml |
| 110 | + |
| 111 | +Run `ecr-public-creds-helper-for-k8s` in your Kubernetes cluster. |
| 112 | + |
| 113 | +```shell |
| 114 | +$ kubectl apply -f cronjob.yaml |
| 115 | +cronjob.batch/ecr-public-creds-helper created |
| 116 | +``` |
| 117 | + |
| 118 | +### Step 6 - (Optional but recommended) Run Job manually |
| 119 | + |
| 120 | +Create an initial auth token manually to let your pods use it without waiting the initial cronjob to be started. Note that `ecs-public-creds-helper-for-k8s` [refreshes the auth token in every 8 hours by default](cronjob.yaml#L8). |
| 121 | + |
| 122 | +```shell |
| 123 | +$ kubectl create job initial-creds-job \ |
| 124 | + -n ecr-public-creds-helper \ |
| 125 | + --from=cronjob/ecr-public-creds-helper |
| 126 | +job.batch/initial-creds-job created |
| 127 | + |
| 128 | +## Check the pod log to make sure it works as expected |
| 129 | +$ export POD_NAME=$(kubectl get pods --selector=job-name=initial-creds-job -n ecr-public-creds-helper -o jsonpath='{.items[0].metadata.name}') |
| 130 | + |
| 131 | +$ echo ${POD_NAME} |
| 132 | +initial-creds-job-r4fbp # you'll see something like this |
| 133 | + |
| 134 | +$ kubectl logs ${POD_NAME} -n ecr-public-creds-helper |
| 135 | +### You'll see the same number of lines as the namespaces you specified in the "TARGET_NAMESPACES" in the cronjob.yaml |
| 136 | +secret/ecr-public-token created |
| 137 | +secret/ecr-public-token created |
| 138 | +secret/ecr-public-token created |
| 139 | + |
| 140 | +$ kubectl delete job initial-creds-job -n ecr-public-creds-helper |
| 141 | +job.batch "initial-creds-job" deleted |
| 142 | +``` |
| 143 | + |
| 144 | +## Use auth tokens in Pods |
| 145 | + |
| 146 | +Now your pod can use the auth token (Kubernetes secret) created by `ecr-public-creds-helper-for-k8s` to pull public container images as an authenticated user from Amazon ECR Public registries. |
| 147 | + |
| 148 | +You can reference the auth token from your pods like: |
| 149 | + |
| 150 | +```yaml |
| 151 | +apiVersion: v1 |
| 152 | +kind: Pod |
| 153 | +# ~ snip ~ |
| 154 | +spec: |
| 155 | +# ~ snip ~ |
| 156 | + imagePullSecrets: |
| 157 | + - name: ecr-public-token |
| 158 | +# ~ snip ~ |
| 159 | +``` |
| 160 | + |
| 161 | +See also [examples/pod.yaml](examples/pod.yaml) for full example. |
| 162 | + |
| 163 | +If you don't want to add `imagePullSecrets` in each PodSpec, you may want to see the comments in the [entrypoint.sh](entrypoint.sh#L18). |
| 164 | + |
| 165 | +## Security |
| 166 | + |
| 167 | +See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. |
| 168 | + |
| 169 | +## License |
| 170 | + |
| 171 | +Licensed under the MIT-0 License. See the [LICENSE](LICENSE) file. |
0 commit comments