Skip to content

Terraform module for deploying a Domino on AKS

License

Notifications You must be signed in to change notification settings

dominodatalab/terraform-azure-aks

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

64 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

terraform-azure-aks

Terraform module for deploying a Domino on AKS

Use

Create a Domino development AKS cluster

module "aks_cluster" {
  source  = "github.com/dominodatalab/terraform-azure-aks"

  cluster = "cluster-name"
}

Manual Deploy

For new projects, the following needs to be done only once for the workspace.

  1. az login
  2. terraform init
  3. terraform workspace new [cluster-name]

Run the Terraform deployment

  1. export TF_VAR_service_principal_name=<service-principal-appid>
  2. export TF_VAR_service_principal_secret=<service-principal-password>
  3. terraform apply -auto-approve

Access AKS cluster

  1. az aks get-credentials --resource-group [cluster-name] --name [cluster-name]

Troubleshooting

  1. No access to Azure backend store In this case you would need to override the backend configuration. This can be done via the command line:
terraform init -backend-config="storage_account_name=<YourAzureStorageAccountName>" -backend-config="container_name=tfstate" -backend-config="access_key=<YourStorageAccountAccessKey>" -backend-config="key=codelab.microsoft.tfstate"

Development

Please submit any feature enhancements, bug fixes, or ideas via pull requests or issues.

Requirements

Name Version
terraform >= 1.3.0
azurerm ~> 3.45
random ~> 3.1

Providers

Name Version
azurerm ~> 3.45
random ~> 3.1

Modules

Name Source Version
domino_acr_ep ./modules/private_endpoint n/a
domino_blob_ep ./modules/private_endpoint n/a
domino_shared_ep ./modules/private_endpoint n/a

Resources

Name Type
azurerm_container_registry.domino resource
azurerm_federated_identity_credential.hephaestus resource
azurerm_federated_identity_credential.importer resource
azurerm_kubernetes_cluster.aks resource
azurerm_kubernetes_cluster_node_pool.aks resource
azurerm_log_analytics_solution.logs resource
azurerm_log_analytics_workspace.logs resource
azurerm_monitor_diagnostic_setting.control_plane resource
azurerm_private_dns_zone.acr_private_dns_zone resource
azurerm_private_dns_zone.aks_private_dns_zone resource
azurerm_private_dns_zone.blob_private_dns_zone resource
azurerm_private_dns_zone.shared_private_dns_zone resource
azurerm_private_dns_zone_virtual_network_link.private_dns_zone_acr_vnet_link resource
azurerm_private_dns_zone_virtual_network_link.private_dns_zone_aks_vnet_link resource
azurerm_private_dns_zone_virtual_network_link.private_dns_zone_blob_vnet_link resource
azurerm_private_dns_zone_virtual_network_link.private_dns_zone_shared_vnet_link resource
azurerm_role_assignment.aks_domino_acr resource
azurerm_role_assignment.aks_domino_private_acr resource
azurerm_role_assignment.aks_domino_shared resource
azurerm_role_assignment.aks_file_share_contributor resource
azurerm_role_assignment.aks_network resource
azurerm_role_assignment.hephaestus_acr resource
azurerm_role_assignment.identity_assign_pdnsz resource
azurerm_role_assignment.identity_assign_rg resource
azurerm_role_assignment.identity_assign_vnet resource
azurerm_storage_account.domino resource
azurerm_storage_account.domino_shared resource
azurerm_storage_account_network_rules.domino_blob_rules resource
azurerm_storage_account_network_rules.domino_shared_rules resource
azurerm_storage_container.domino_containers resource
azurerm_storage_share.shared_store resource
azurerm_user_assigned_identity.aks_assigned_identity resource
azurerm_user_assigned_identity.hephaestus resource
random_id.log_analytics_workspace_name_suffix resource
azurerm_kubernetes_service_versions.selected data source
azurerm_resource_group.aks data source
azurerm_subnet.aks_subnet data source
azurerm_subscription.current data source
azurerm_virtual_network.aks_vnet data source

Inputs

Name Description Type Default Required
additional_node_pools additional node pools
map(object({
enable_node_public_ip = optional(bool, false)
vm_size = string
zones = list(string)
node_labels = map(string)
node_os = optional(string, "AzureLinux")
node_taints = optional(list(string), [])
enable_auto_scaling = optional(bool, true)
min_count = optional(number, 0)
max_count = number
initial_count = optional(number, 0)
max_pods = optional(number, 30)
os_disk_size_gb = optional(number, 128)
}))
{} no
aks_subnet_name Subnet name for ACR/AKS, required when either private_acr_enabled or private_cluster_enabled is set to true. string null no
aks_vnet_name VNet name for ACR/AKS, required when either private_acr_enabled or private_cluster_enabled is set to true. string null no
aks_vnet_rg_name VNet Resource Groupe name for ACR/AKS, required when either private_acr_enabled or private_cluster_enabled is set to true. string null no
api_server_authorized_ip_ranges The IP ranges to whitelist for incoming traffic to the masters list(string) null no
cluster_sku_tier The Domino cluster SKU (defaults to Free) string null no
cni_overlay_enabled Flag to determine whether to use overlay network settings bool false no
containers storage containers to create
map(object({
container_access_type = string
}))
{
"backups": {
"container_access_type": "private"
},
"projects": {
"container_access_type": "private"
}
}
no
deploy_id Domino Deployment ID. string n/a yes
dns_service_ip IP address assigned to the Kubernetes DNS service, used when CNI Overlay is enabled string "100.97.0.10" no
kubeconfig_output_path kubeconfig path string n/a yes
kubernetes_nat_gateway Managed NAT Gateway configuration
object({
idle_timeout_in_minutes = optional(number, 4)
managed_outbound_ip_count = number
}
)
null no
kubernetes_version Optional Kubernetes version to provision. Allows partial input (e.g. 1.18) which is then chosen from azurerm_kubernetes_service_versions. string null no
log_analytics_workspace_sku log analytics sku string "PerGB2018" no
namespaces Namespace that are used for generating the service account bindings object({ platform = string, compute = string }) n/a yes
node_pools default node pools
object({
compute = object({
enable_node_public_ip = optional(bool, false)
vm_size = optional(string, "Standard_D8s_v4")
zones = optional(list(string), ["1", "2", "3"])
node_labels = optional(map(string), {
"dominodatalab.com/node-pool" = "default"
})
node_os = optional(string, "AzureLinux")
node_taints = optional(list(string), [])
enable_auto_scaling = optional(bool, true)
min_count = optional(number, 0)
max_count = optional(number, 10)
initial_count = optional(number, 1)
max_pods = optional(number, 30)
os_disk_size_gb = optional(number, 128)
}),
platform = object({
enable_node_public_ip = optional(bool, false)
vm_size = optional(string, "Standard_D8s_v4")
zones = optional(list(string), ["1", "2", "3"])
node_labels = optional(map(string), {
"dominodatalab.com/node-pool" = "platform"
})
node_os = optional(string, "AzureLinux")
node_taints = optional(list(string), [])
enable_auto_scaling = optional(bool, true)
min_count = optional(number, 1)
max_count = optional(number, 3)
initial_count = optional(number, 1)
max_pods = optional(number, 60)
os_disk_size_gb = optional(number, 128)
}),
gpu = object({
enable_node_public_ip = optional(bool, false)
vm_size = optional(string, "Standard_NC6s_v3")
zones = optional(list(string), [])
node_labels = optional(map(string), {
"dominodatalab.com/node-pool" = "default-gpu"
"nvidia.com/gpu" = "true"
})
node_os = optional(string, "AzureLinux")
node_taints = optional(list(string), [
"nvidia.com/gpu=true:NoExecute"
])
enable_auto_scaling = optional(bool, true)
min_count = optional(number, 0)
max_count = optional(number, 1)
initial_count = optional(number, 0)
max_pods = optional(number, 30)
os_disk_size_gb = optional(number, 128)
})
system = object({
enable_node_public_ip = optional(bool, false)
vm_size = optional(string, "Standard_DS4_v2")
zones = optional(list(string), ["1", "2", "3"])
node_labels = optional(map(string), {})
node_os = optional(string, "AzureLinux")
node_taints = optional(list(string), [])
enable_auto_scaling = optional(bool, true)
min_count = optional(number, 1)
max_count = optional(number, 6)
initial_count = optional(number, 1)
max_pods = optional(number, 60)
os_disk_size_gb = optional(number, 128)
})
})
{
"compute": {},
"gpu": {},
"platform": {},
"system": {}
}
no
pod_cidr CIDR block for Kubernetes pods, used when CNI Overlay is enabled string "192.168.0.0/16" no
private_acr_enabled Flag to determine whether to deploy a private ACR bool false no
private_cluster_enabled Flag to determine whether to deploy a private AKS bool false no
private_cluster_public_fqdn_enabled Flag to determine whether to use a public FQDN when deploying a private AKS cluster bool null no
registry_tier registry tier string "Standard" no
resource_group Name or id of optional pre-existing resource group to install AKS in string n/a yes
service_cidr CIDR block for Kubernetes services, used when CNI Overlay is enabled string "100.97.0.0/16" no
storage_account_replication_type storage replication string "LRS" no
storage_account_tier storage account tier string "Standard" no
tags Tags to apply to resources map(string) {} no

Outputs

Name Description
aks_identity AKS managed identity
blob_dns_zone_name blob dns zone name
containers storage details
domino_acr Azure Container Registry details
oidc_issuer_url OIDC issuer url
private_cluster_enabled Flag to determine if AKS is private or public
shared_storage_account shared storage account
storage_account storage account
workload_identities service identities