Skip to content

Latest commit

 

History

History
153 lines (132 loc) · 18.5 KB

README.md

File metadata and controls

153 lines (132 loc) · 18.5 KB
Raw

terraform-azure-aks

Terraform module for deploying a Domino on AKS

Use

Create a Domino development AKS cluster

module "aks_cluster" {
  source  = "github.com/dominodatalab/terraform-azure-aks"

  cluster = "cluster-name"
}

Manual Deploy

For new projects, the following needs to be done only once for the workspace.

  1. az login
  2. terraform init
  3. terraform workspace new [cluster-name]

Run the Terraform deployment

  1. export TF_VAR_service_principal_name=<service-principal-appid>
  2. export TF_VAR_service_principal_secret=<service-principal-password>
  3. terraform apply -auto-approve

Access AKS cluster

  1. az aks get-credentials --resource-group [cluster-name] --name [cluster-name]

Troubleshooting

  1. No access to Azure backend store In this case you would need to override the backend configuration. This can be done via the command line:
terraform init -backend-config="storage_account_name=<YourAzureStorageAccountName>" -backend-config="container_name=tfstate" -backend-config="access_key=<YourStorageAccountAccessKey>" -backend-config="key=codelab.microsoft.tfstate"

Development

Please submit any feature enhancements, bug fixes, or ideas via pull requests or issues.

Requirements

Name Version
terraform >= 1.3.0
azurerm ~> 3.45
random ~> 3.1

Providers

Name Version
azurerm ~> 3.45
random ~> 3.1

Modules

Name Source Version
domino_acr_ep ./modules/private_endpoint n/a
domino_blob_ep ./modules/private_endpoint n/a
domino_shared_ep ./modules/private_endpoint n/a

Resources

Name Type
azurerm_container_registry.domino resource
azurerm_federated_identity_credential.hephaestus resource
azurerm_federated_identity_credential.importer resource
azurerm_kubernetes_cluster.aks resource
azurerm_kubernetes_cluster_node_pool.aks resource
azurerm_log_analytics_solution.logs resource
azurerm_log_analytics_workspace.logs resource
azurerm_monitor_diagnostic_setting.control_plane resource
azurerm_private_dns_zone.acr_private_dns_zone resource
azurerm_private_dns_zone.aks_private_dns_zone resource
azurerm_private_dns_zone.blob_private_dns_zone resource
azurerm_private_dns_zone.shared_private_dns_zone resource
azurerm_private_dns_zone_virtual_network_link.private_dns_zone_acr_vnet_link resource
azurerm_private_dns_zone_virtual_network_link.private_dns_zone_aks_vnet_link resource
azurerm_private_dns_zone_virtual_network_link.private_dns_zone_blob_vnet_link resource
azurerm_private_dns_zone_virtual_network_link.private_dns_zone_shared_vnet_link resource
azurerm_role_assignment.aks_domino_acr resource
azurerm_role_assignment.aks_domino_private_acr resource
azurerm_role_assignment.aks_domino_shared resource
azurerm_role_assignment.aks_file_share_contributor resource
azurerm_role_assignment.aks_network resource
azurerm_role_assignment.hephaestus_acr resource
azurerm_role_assignment.identity_assign_pdnsz resource
azurerm_role_assignment.identity_assign_rg resource
azurerm_role_assignment.identity_assign_vnet resource
azurerm_storage_account.domino resource
azurerm_storage_account.domino_shared resource
azurerm_storage_account_network_rules.domino_blob_rules resource
azurerm_storage_account_network_rules.domino_shared_rules resource
azurerm_storage_container.domino_containers resource
azurerm_storage_share.shared_store resource
azurerm_user_assigned_identity.aks_assigned_identity resource
azurerm_user_assigned_identity.hephaestus resource
random_id.log_analytics_workspace_name_suffix resource
azurerm_kubernetes_service_versions.selected data source
azurerm_resource_group.aks data source
azurerm_subnet.aks_subnet data source
azurerm_subscription.current data source
azurerm_virtual_network.aks_vnet data source

Inputs

Name Description Type Default Required
additional_node_pools additional node pools 
map(object({
    enable_node_public_ip = optional(bool, false)
    vm_size               = string
    zones                 = list(string)
    node_labels           = map(string)
    node_os               = optional(string, "AzureLinux")
    node_taints           = optional(list(string), [])
    enable_auto_scaling   = optional(bool, true)
    min_count             = optional(number, 0)
    max_count             = number
    initial_count         = optional(number, 0)
    max_pods              = optional(number, 30)
    os_disk_size_gb       = optional(number, 128)
  }))
 {} no
aks_subnet_name Subnet name for ACR/AKS, required when either private_acr_enabled or private_cluster_enabled is set to true. string null no
aks_vnet_name VNet name for ACR/AKS, required when either private_acr_enabled or private_cluster_enabled is set to true. string null no
aks_vnet_rg_name VNet Resource Groupe name for ACR/AKS, required when either private_acr_enabled or private_cluster_enabled is set to true. string null no
api_server_authorized_ip_ranges The IP ranges to whitelist for incoming traffic to the masters list(string) null no
cluster_sku_tier The Domino cluster SKU (defaults to Free) string null no
cni_overlay_enabled Flag to determine whether to use overlay network settings bool false no
containers storage containers to create 
map(object({
    container_access_type = string
  }))
 
{
  "backups": {
    "container_access_type": "private"
  },
  "projects": {
    "container_access_type": "private"
  }
}
 no
deploy_id Domino Deployment ID. string n/a yes
dns_service_ip IP address assigned to the Kubernetes DNS service, used when CNI Overlay is enabled string "100.97.0.10" no
kubeconfig_output_path kubeconfig path string n/a yes
kubernetes_nat_gateway Managed NAT Gateway configuration 
object({
    idle_timeout_in_minutes   = optional(number, 4)
    managed_outbound_ip_count = number
    }
  )
 null no
kubernetes_version Optional Kubernetes version to provision. Allows partial input (e.g. 1.18) which is then chosen from azurerm_kubernetes_service_versions. string null no
log_analytics_workspace_sku log analytics sku string "PerGB2018" no
namespaces Namespace that are used for generating the service account bindings object({ platform = string, compute = string }) n/a yes
node_pools default node pools 
object({
    compute = object({
      enable_node_public_ip = optional(bool, false)
      vm_size               = optional(string, "Standard_D8s_v4")
      zones                 = optional(list(string), ["1", "2", "3"])
      node_labels = optional(map(string), {
        "dominodatalab.com/node-pool" = "default"
      })
      node_os             = optional(string, "AzureLinux")
      node_taints         = optional(list(string), [])
      enable_auto_scaling = optional(bool, true)
      min_count           = optional(number, 0)
      max_count           = optional(number, 10)
      initial_count       = optional(number, 1)
      max_pods            = optional(number, 30)
      os_disk_size_gb     = optional(number, 128)
    }),
    platform = object({
      enable_node_public_ip = optional(bool, false)
      vm_size               = optional(string, "Standard_D8s_v4")
      zones                 = optional(list(string), ["1", "2", "3"])
      node_labels = optional(map(string), {
        "dominodatalab.com/node-pool" = "platform"
      })
      node_os             = optional(string, "AzureLinux")
      node_taints         = optional(list(string), [])
      enable_auto_scaling = optional(bool, true)
      min_count           = optional(number, 1)
      max_count           = optional(number, 3)
      initial_count       = optional(number, 1)
      max_pods            = optional(number, 60)
      os_disk_size_gb     = optional(number, 128)
    }),
    gpu = object({
      enable_node_public_ip = optional(bool, false)
      vm_size               = optional(string, "Standard_NC6s_v3")
      zones                 = optional(list(string), [])
      node_labels = optional(map(string), {
        "dominodatalab.com/node-pool" = "default-gpu"
        "nvidia.com/gpu"              = "true"
      })
      node_os = optional(string, "AzureLinux")
      node_taints = optional(list(string), [
        "nvidia.com/gpu=true:NoExecute"
      ])
      enable_auto_scaling = optional(bool, true)
      min_count           = optional(number, 0)
      max_count           = optional(number, 1)
      initial_count       = optional(number, 0)
      max_pods            = optional(number, 30)
      os_disk_size_gb     = optional(number, 128)
    })
    system = object({
      enable_node_public_ip = optional(bool, false)
      vm_size               = optional(string, "Standard_DS4_v2")
      zones                 = optional(list(string), ["1", "2", "3"])
      node_labels           = optional(map(string), {})
      node_os               = optional(string, "AzureLinux")
      node_taints           = optional(list(string), [])
      enable_auto_scaling   = optional(bool, true)
      min_count             = optional(number, 1)
      max_count             = optional(number, 6)
      initial_count         = optional(number, 1)
      max_pods              = optional(number, 60)
      os_disk_size_gb       = optional(number, 128)
    })
  })
 
{
  "compute": {},
  "gpu": {},
  "platform": {},
  "system": {}
}
 no
pod_cidr CIDR block for Kubernetes pods, used when CNI Overlay is enabled string "192.168.0.0/16" no
private_acr_enabled Flag to determine whether to deploy a private ACR bool false no
private_cluster_enabled Flag to determine whether to deploy a private AKS bool false no
private_cluster_public_fqdn_enabled Flag to determine whether to use a public FQDN when deploying a private AKS cluster bool null no
registry_tier registry tier string "Standard" no
resource_group Name or id of optional pre-existing resource group to install AKS in string n/a yes
service_cidr CIDR block for Kubernetes services, used when CNI Overlay is enabled string "100.97.0.0/16" no
storage_account_replication_type storage replication string "LRS" no
storage_account_tier storage account tier string "Standard" no
tags Tags to apply to resources map(string) {} no

Outputs

Name Description
aks_identity AKS managed identity
blob_dns_zone_name blob dns zone name
containers storage details
domino_acr Azure Container Registry details
oidc_issuer_url OIDC issuer url
private_cluster_enabled Flag to determine if AKS is private or public
shared_storage_account shared storage account
storage_account storage account
workload_identities service identities