Name	Name	Last commit message	Last commit date
parent directory ..
ci	ci
crds	crds
templates	templates
upgrade	upgrade
.helmignore	.helmignore
Chart.yaml	Chart.yaml
README.md	README.md
test.yaml	test.yaml
values.yaml	values.yaml
webhookconfig.yaml	webhookconfig.yaml

App Mesh Controller

App Mesh controller Helm chart for Kubernetes

Note: If you wish to use App Mesh preview features, please refer to our preview version instructions.

Prerequisites

Kubernetes >= 1.14
IAM permissions (see below)

Installing the Chart

Note: AppMesh controller v1.0.0+ is backwards incompatible with old versions(e.g. v0.5.0). If you're running an older version of App Mesh controller, please go to the upgrade section below before you proceed. If you are unsure, please run the appmesh-controller/upgrade/pre_upgrade_check.sh script to check if your cluster can be upgraded

Add the EKS repository to Helm:

helm repo add eks https://aws.github.io/eks-charts

Install the App Mesh CRDs:

kubectl apply -k "github.com/aws/eks-charts/stable/appmesh-controller//crds?ref=master"

Create namespace

kubectl create ns appmesh-system

The controller runs on the worker nodes, so it needs access to the AWS App Mesh / Cloud Map resources via IAM permissions. The IAM permissions can either be setup via IAM roles for service account or can be attached directly to the worker node IAM roles.

Setup IAM Role for Service Account

export CLUSTER_NAME=<eks-cluster-name>
export AWS_REGION=<aws-region e.g. us-east-1>
export AWS_ACCOUNT_ID=<AWS account ID>

Enable IAM OIDC provider

eksctl utils associate-iam-oidc-provider --region=$AWS_REGION \
    --cluster=$CLUSTER_NAME \
    --approve

Download the IAM policy for AWS App Mesh Kubernetes Controller

curl -o controller-iam-policy.json https://raw.githubusercontent.com/aws/aws-app-mesh-controller-for-k8s/master/config/iam/controller-iam-policy.json

Create an IAM policy called AWSAppMeshK8sControllerIAMPolicy

aws iam create-policy \
    --policy-name AWSAppMeshK8sControllerIAMPolicy \
    --policy-document file://controller-iam-policy.json

Take note of the policy ARN that is returned

Create an IAM role for service account for the App Mesh Kubernetes controller, use the ARN from the step above

Note: if you deleted serviceaccount in the appmesh-system namespace, you will need to delete and re-create iamserviceaccount. eksctl does not override the iamserviceaccount correctly (see this issue)

eksctl create iamserviceaccount --cluster $CLUSTER_NAME \
    --namespace appmesh-system \
    --name appmesh-controller \
    --attach-policy-arn arn:aws:iam::$AWS_ACCOUNT_ID:policy/AWSAppMeshK8sControllerIAMPolicy  \
    --override-existing-serviceaccounts \
    --approve

Deploy appmesh-controller

Note: To enable mTLS via SDS(SPIRE), please set "sds.enabled=true".

helm upgrade -i appmesh-controller eks/appmesh-controller \
    --namespace appmesh-system \
    --set region=$AWS_REGION \
    --set serviceAccount.create=false \
    --set serviceAccount.name=appmesh-controller

The configuration section lists the parameters that can be configured during installation.

Note: When using IRSA, make sure the Envoy proxies have the following IAM policies attached for Envoy to authenticate with AWS App Mesh and fetch it's configuration

https://raw.githubusercontent.com/aws/aws-app-mesh-controller-for-k8s/master/config/iam/envoy-iam-policy.json

Setup IAM permissions manually on worker nodes

If not setting up IAM role for service account, apply the IAM policies to your worker nodes:

Controller IAM policy

https://raw.githubusercontent.com/aws/aws-app-mesh-controller-for-k8s/master/config/iam/controller-iam-policy.json

Envoy IAM policy

https://raw.githubusercontent.com/aws/aws-app-mesh-controller-for-k8s/master/config/iam/envoy-iam-policy.json

Deploy appmesh-controller

helm upgrade -i appmesh-controller eks/appmesh-controller \
    --namespace appmesh-system

The configuration section lists the parameters that can be configured during installation.

Installation on EKS with Fargate

export CLUSTER_NAME=<eks-cluster-name>
export AWS_REGION=<aws-region e.g. us-east-1>
export AWS_ACCOUNT_ID=<AWS account ID>

Create namespace

kubectl create ns appmesh-system

Setup EKS Fargate profile

eksctl create fargateprofile --cluster $CLUSTER_NAME --namespace appmesh-system

Enable IAM OIDC provider

eksctl utils associate-iam-oidc-provider --region=$AWS_REGION --cluster=$CLUSTER_NAME --approve

Download the IAM policy for AWS App Mesh Kubernetes Controller

curl -o controller-iam-policy.json https://raw.githubusercontent.com/aws/aws-app-mesh-controller-for-k8s/master/config/iam/controller-iam-policy.json

Create an IAM policy called AWSAppMeshK8sControllerIAMPolicy

aws iam create-policy \
    --policy-name AWSAppMeshK8sControllerIAMPolicy \
    --policy-document file://controller-iam-policy.json

Take note of the policy ARN that is returned

Create an IAM role for service account for the App Mesh Kubernetes controller, use the ARN from the step above

Note: if you deleted serviceaccount in the appmesh-system namespace, you will need to delete and re-create iamserviceaccount. eksctl does not override the iamserviceaccount correctly (see this issue)

eksctl create iamserviceaccount --cluster $CLUSTER_NAME \
    --namespace appmesh-system \
    --name appmesh-controller \
    --attach-policy-arn arn:aws:iam::$AWS_ACCOUNT_ID:policy/AWSAppMeshK8sControllerIAMPolicy  \
    --override-existing-serviceaccounts \
    --approve

Deploy appmesh-controller

Note: mTLS via SDS(SPIRE) is not supported on Fargate.

helm upgrade -i appmesh-controller eks/appmesh-controller \
    --namespace appmesh-system \
    --set region=$AWS_REGION \
    --set serviceAccount.create=false \
    --set serviceAccount.name=appmesh-controller

Upgrade

This section will assist you in upgrading the appmesh-controller from <=v0.5.0 version to >=v1.0.0 version.

You can either build new CRDs from scratch or migrate existing CRDs to the new schema. Please refer to the documentation here for the new API spec. Also, you can find several examples here with v1beta2 spec to help you get started.

Starting v1.0.0, Mesh resource supports namespaceSelectors, where you can either select namespace based on labels (recommended option) or select all namespaces. To select a namespace in a Mesh, you will need to define namespaceSelector:

apiVersion: appmesh.k8s.aws/v1beta2
kind: Mesh
metadata:
  name: <mesh-name>
spec:
  namespaceSelector:
    matchLabels:
      mesh: <mesh-name> // any string value

Note: If you set namespaceSelector: {}, mesh will select all the namespace in your cluster. Labels on your namespace spec is a no-op when selecting all namespaces.

In the namespace spec, you will need to add a label mesh: <mesh-name>. Here's a sample namespace spec:

apiVersion: v1
kind: Namespace
metadata:
  name: ns
  labels:
    mesh: <mesh-name>
    appmesh.k8s.aws/sidecarInjectorWebhook: enabled

For more examples, please refer to the walkthroughs here. If you don't find an example that fits your use-case, please read the API spec here. If you find an issue in the documentation or the examples, please open an issue and we'll help resolve it.

Upgrade without preserving old App Mesh resources

# Keep old App Mesh controller running, it is responsible to cleanup App Mesh resources in AWS
# Delete all existing App Mesh custom resources (CRs)
kubectl delete virtualservices --all --all-namespaces
kubectl delete virtualnodes --all --all-namespaces
kubectl delete meshes --all --all-namespaces

# Delete all existing App Mesh CRDs
kubectl delete customresourcedefinition/virtualservices.appmesh.k8s.aws
kubectl delete customresourcedefinition/virtualnodes.appmesh.k8s.aws
kubectl delete customresourcedefinition/meshes.appmesh.k8s.aws
# Note: If a CRD stuck in deletion, it means there still exists some App Mesh custom resources, please check and delete them.

# Delete App Mesh controller
helm delete appmesh-controller -n appmesh-system

# Delete App Mesh injector
helm delete appmesh-inject -n appmesh-system

Run the appmesh-controller/upgrade/pre_upgrade_check.sh script and make sure it passes before you proceed

Now you can proceed with the installation steps described above

Upgrade preserving old App Mesh resources

# Save manifests of all existing App Mesh custom resources
kubectl get virtualservices --all-namespaces -o yaml > virtualservices.yaml
kubectl get virtualnodes --all-namespaces -o yaml > virtualnodes.yaml
kubectl get meshes --all-namespaces -o yaml > meshes.yaml

# Delete App Mesh controller, so it won’t clean up App Mesh resources in AWS while we deleting App Mesh CRs later.
helm delete appmesh-controller -n appmesh-system

# Delete App Mesh injector.
helm delete appmesh-inject -n appmesh-system

# Remove finalizers from all existing App Mesh CRs. Otherwise, you won’t be able to delete them

# To remove the finalizers, you could kubectl edit resource, and delete the finalizers attribute from the spec or run the following command to override finalizers. e.g for virtualnodes
# kubectl get virtualnodes --all-namespaces -o=jsonpath='{range .items[*]}{.metadata.namespace}{"\t"}{.metadata.name}{"\n"}{end}' | xargs -n2 sh -c 'kubectl patch virtualnode/$1 -n $0 -p '\''{"metadata":{"finalizers":null}}'\'' --type=merge'

# Alternatively, you could modify one resource at a time using
# kubectl get <RESOURCE_TYPE> <RESOURCE_NAME> -n <NAMESPACE> -o=json | jq '.metadata.finalizers = null' | kubectl apply -f -

# Delete all existing App Mesh CRs:
kubectl delete virtualservices --all --all-namespaces
kubectl delete virtualnodes --all --all-namespaces
kubectl delete meshes --all --all-namespaces

# Delete all existing App Mesh CRDs.
kubectl delete customresourcedefinition/virtualservices.appmesh.k8s.aws
kubectl delete customresourcedefinition/virtualnodes.appmesh.k8s.aws
kubectl delete customresourcedefinition/meshes.appmesh.k8s.aws
# Note: If CRDs are stuck in deletion, it means there still exists some App Mesh CRs, please check and delete them.

Run the appmesh-controller/upgrade/pre_upgrade_check.sh script and make sure it passes before you proceed

Translate the saved old YAML manifests using v1beta1 App Mesh CRD into v1beta2 App Mesh CRD format. Please refer to CRD types ( https://github.com/aws/aws-app-mesh-controller-for-k8s/tree/master/config/crd/bases) and Go types (https://github.com/aws/aws-app-mesh-controller-for-k8s/tree/master/apis/appmesh/v1beta2) for the CRD Documentation. Samples applications are in the repo https://github.com/aws/aws-app-mesh-examples for reference.

Note: Please specify the current appmesh resource names in the awsName field of the translated specs.

Install the appmesh-controller, label the namespace with values that mesh is selecting on and apply the translated manifest

Upgrade from prior script installation

If you've installed the App Mesh controllers with scripts, you can remove the controllers with the steps below.

# remove injector objects
kubectl delete ns appmesh-inject
kubectl delete ClusterRoleBinding aws-app-mesh-inject-binding
kubectl delete ClusterRole aws-app-mesh-inject-cr
kubectl delete  MutatingWebhookConfiguration aws-app-mesh-inject

# remove controller objects
kubectl delete ns appmesh-system
kubectl delete ClusterRoleBinding app-mesh-controller-binding
kubectl delete ClusterRole app-mesh-controller

Run the appmesh-controller/upgrade/pre_upgrade_check.sh script and make sure it passes before you proceed

For handling the existing custom resources and the CRDs please refer to either of the previous upgrade sections as relevant.

Uninstalling the Chart

To uninstall/delete the appmesh-controller deployment:

$ helm delete appmesh-controller -n appmesh-system

The command removes all the Kubernetes components associated with the chart and deletes the release.

Configuration

The following tables lists the configurable parameters of the chart and their default values.

Parameter	Description	Default
`image.repository`	image repository	`602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/appmesh-controller`
`image.tag`	image tag	`<VERSION>`
`image.pullPolicy`	image pull policy	`IfNotPresent`
`log.level`	controller log level, possible values are `info` and `debug`	`info`
`sds.enabled`	If `true`, SDS will be enabled in Envoy	`false`
`sds.udsPath`	Unix Domain Socket Path of the SDS Provider(SPIRE in the current release)	`/run/spire/sockets/agent.sock`
`resources.requests/cpu`	pod CPU request	`100m`
`resources.requests/memory`	pod memory request	`64Mi`
`resources.limits/cpu`	pod CPU limit	`2000m`
`resources.limits/memory`	pod memory limit	`1Gi`
`affinity`	node/pod affinities	None
`nodeSelector`	node labels for pod assignment	`{}`
`podAnnotations`	annotations to add to each pod	`{}`
`podLabels`	labels to add to each pod	`{}`
`tolerations`	list of node taints to tolerate	`[]`
`rbac.create`	if `true`, create and use RBAC resources	`true`
`rbac.pspEnabled`	If `true`, create and use a restricted pod security policy	`false`
`serviceAccount.annotations`	optional annotations to add to service account	None
`serviceAccount.create`	If `true`, create a new service account	`true`
`serviceAccount.name`	Service account to be used	None
`sidecar.image.repository`	Envoy image repository. If you override with non-Amazon built Envoy image, you will need to test/ensure it works with the App Mesh	`840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy`
`sidecar.image.tag`	Envoy image tag	`<VERSION>`
`sidecar.logLevel`	Envoy log level	`info`
`sidecar.envoyAdminAccessPort`	Envoy Admin Access Port	`9901`
`sidecar.envoyAdminAccessLogFile`	Envoy Admin Access Log File	`/tmp/envoy_admin_access.log`
`sidecar.resources.requests`	Envoy container resource requests	`requests: cpu 10m memory 32Mi`
`sidecar.resources.limits`	Envoy container resource limits	`limits: cpu "" memory ""`
`sidecar.lifecycleHooks.preStopDelay`	Envoy container PreStop Hook Delay Value	`20s`
`sidecar.probes.readinessProbeInitialDelay`	Envoy container Readiness Probe Initial Delay	`1s`
`sidecar.probes.readinessProbePeriod`	Envoy container Readiness Probe Period	`10s`
`init.image.repository`	Route manager image repository	`840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-proxy-route-manager`
`init.image.tag`	Route manager image tag	`<VERSION>`
`stats.tagsEnabled`	If `true`, Envoy should include app-mesh tags	`false`
`stats.statsdEnabled`	If `true`, Envoy should publish stats to statsd endpoint @ 127.0.0.1:8125	`false`
`stats.statsdAddress`	DogStatsD daemon IP address	`127.0.0.1`
`stats.statsdPort`	DogStatsD daemon port	`8125`
`cloudMapCustomHealthCheck.enabled`	If `true`, CustomHealthCheck will be enabled for CloudMap Services	`false`
`cloudMapDNS.ttl`	Sets CloudMap DNS TTL	`300`
`tracing.enabled`	If `true`, Envoy will be configured with tracing	`false`
`tracing.provider`	The tracing provider can be x-ray, jaeger or datadog	`x-ray`
`tracing.address`	Jaeger or Datadog agent server address (ignored for X-Ray)	`appmesh-jaeger.appmesh-system`
`tracing.port`	Jaeger or Datadog agent port (ignored for X-Ray)	`9411`
`enableCertManager`	Enable Cert-Manager	`false`
`xray.image.repository`	X-Ray image repository	`amazon/aws-xray-daemon`
`xray.image.tag`	X-Ray image tag	`latest`
`accountId`	AWS Account ID for the Kubernetes cluster	None
`env`	environment variables to be injected into the appmesh-controller pod	`{}`
`livenessProbe`	Liveness probe settings for the controller	(see `values.yaml`)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

appmesh-controller

appmesh-controller

README.md

App Mesh Controller

Prerequisites

Installing the Chart

Setup IAM Role for Service Account

Setup IAM permissions manually on worker nodes

Installation on EKS with Fargate

Upgrade

Upgrade without preserving old App Mesh resources

Upgrade preserving old App Mesh resources

Upgrade from prior script installation

Uninstalling the Chart

Configuration

Files

appmesh-controller

Directory actions

More options

Directory actions

More options

Latest commit

History

appmesh-controller

Folders and files

parent directory

README.md

App Mesh Controller

Prerequisites

Installing the Chart

Setup IAM Role for Service Account

Setup IAM permissions manually on worker nodes

Installation on EKS with Fargate

Upgrade

Upgrade without preserving old App Mesh resources

Upgrade preserving old App Mesh resources

Upgrade from prior script installation

Uninstalling the Chart

Configuration