Set default toleration value to avoid `error connecting to provider "aws"`

[riosje]

Set a default Toleration value

There are several issues including this one #266 where the users get the error error connecting to provider "aws" due to the POD tolerations.

I think that it should be set by default [{"operator": "Exists"}] as it is on secrets-store-csi-driver

This will save a ton of debugging time to a lot of people.

Update the troubleshooting section with this issue is also a good option, in case there is not possible to update the default value on the helm chart.

[marquesmateus93]

In my case, I have a Node Group with Taints, and the Secrets Storage CSI Driver Provider has not been deployed.
I solve my issue by configuring secrets-store-csi-driver and secrets-store-csi-driver-provider-aws:

secrets-store-csi-driver

local.tf

locals {
  csi-secrets-store = {
    name = lower("${var.prefix_name}-secrets-store-helm-${var.tags["environment"]}")

    tolerations = {
        key       = "MY_KEY"
        operator  = "Exists"
        effect    = "NoSchedule"
    }
  }
}

secrets-store-csi-driver.tf

resource "helm_release" "csi-secrets-store" {
  count       = var.is_enabled ? 1:0

  name        = local.csi-secrets-store.name
  repository  = var.helm.repository
  chart       = var.helm.chart
  namespace   = var.helm.namespace

  set {
    name  = "syncSecret.enabled"
    value = "true"
  }

  dynamic "set" {
    for_each = local.csi-secrets-store.tolerations
    content {
      name  = "linux.tolerations[${count.index}].${set.key}"
      value = set.value
    }
  }
}

secrets-store-csi-driver-provider-aws

local.tf

locals {
  aws-secrets-store = {
    name = lower("${var.prefix_name}-aws-secrets-manager-${var.tags["environment"]}")

    tolerations = {
      key       = "MY_KEY"
      operator  = "Exists"
      effect    = "NoSchedule"
    }
  }
}

aws-secrets-store.tf

resource "helm_release" "aws-secrets-store" {
  count       = var.is_enabled ? 1:0

  name        = local.aws-secrets-store.name
  repository  = var.helm.repository
  chart       = var.helm.chart
  namespace   = var.helm.namespace

  dynamic "set" {
    for_each = local.aws-secrets-store.tolerations
    content {
      name  = "tolerations[${count.index}].${set.key}"
      value = set.value
    }
  }
}