Skip to content

Latest commit

 

History

History
147 lines (105 loc) · 8.94 KB

kubernetes_nodes.md

File metadata and controls

147 lines (105 loc) · 8.94 KB
page_title cozystack_kubernetes_nodes Resource - cozystack
subcategory
description A worker node pool for a Cozystack managed Kubernetes cluster. The pool attaches to its parent cluster by name: cluster names the cozystack_kubernetes object in the same namespace, and the pool itself must be named <cluster>-<pool>. A pool whose name collides with a node group the parent cluster still manages — the default md0, for instance — fails to render on an ownership conflict. The talos, images, and kubelet blocks are left to the chart unless set, and stay out of state when unset, so a pool keeps following the Talos release, schematic, and images that ship with the installed Cozystack version — the same ones its parent cluster follows. A pool cannot report Ready until its workers join, which needs the parent control plane up first, so wait_for_ready on a pool created alongside its cluster will usually spend the whole wait_timeout. Import captures the overrides an existing pool carries. Copy the imported talos, images, kubelet, roles, and gpus values into the configuration before the first apply: an update replaces the whole spec, so an override present on the object but absent from configuration is removed by that apply — the plan shows the removal, but only the configuration can prevent it.

cozystack_kubernetes_nodes (Resource)

A worker node pool for a Cozystack managed Kubernetes cluster. The pool attaches to its parent cluster by name: cluster names the cozystack_kubernetes object in the same namespace, and the pool itself must be named <cluster>-<pool>. A pool whose name collides with a node group the parent cluster still manages — the default md0, for instance — fails to render on an ownership conflict.

The talos, images, and kubelet blocks are left to the chart unless set, and stay out of state when unset, so a pool keeps following the Talos release, schematic, and images that ship with the installed Cozystack version — the same ones its parent cluster follows.

A pool cannot report Ready until its workers join, which needs the parent control plane up first, so wait_for_ready on a pool created alongside its cluster will usually spend the whole wait_timeout.

Import captures the overrides an existing pool carries. Copy the imported talos, images, kubelet, roles, and gpus values into the configuration before the first apply: an update replaces the whole spec, so an override present on the object but absent from configuration is removed by that apply — the plan shows the removal, but only the configuration can prevent it.

Example Usage

resource "cozystack_kubernetes" "cluster" {
  name      = "demo"
  namespace = "tenant-root"

  version = "v1.35"
}

# A pool is named "<cluster>-<pool>".
resource "cozystack_kubernetes_nodes" "gpu" {
  name      = "${cozystack_kubernetes.cluster.name}-gpu"
  namespace = cozystack_kubernetes.cluster.namespace
  cluster   = cozystack_kubernetes.cluster.name

  version       = cozystack_kubernetes.cluster.version
  instance_type = "u1.xlarge"
  disk_size     = "100Gi"
  min_replicas  = 0
  max_replicas  = 4
  roles         = ["gpu"]

  gpus = [
    { name = "nvidia.com/AD102GL_L40S" },
  ]

  kubelet = {
    kube_reserved_memory = "1Gi"
  }
}

Schema

Required

  • cluster (String) Name of the parent cozystack_kubernetes cluster in the same namespace this pool attaches to. A pool cannot be rewired to another cluster in place. Immutable.
  • name (String) Node pool name (metadata.name). Must be <cluster>-<pool>, where <cluster> matches the cluster attribute and <pool> names the pool — demo-md1 for pool md1 of cluster demo. Immutable.
  • namespace (String) Tenant namespace the application is created in. Immutable.

Optional

  • disk_size (String) System disk size for each worker VM. Carries the Talos image, kubelet state, and the containerd image cache.
  • gpus (Attributes List) GPUs to attach to each node (the NVIDIA driver needs at least 4 GiB of memory). (see below for nested schema)
  • images (Attributes) Image overrides for air-gapped or rate-limited registries. Each field follows the chart when unset. (see below for nested schema)
  • instance_type (String) Virtual machine instance type.
  • kubelet (Attributes) Kubelet resource reservations. The reserved CPU and memory fields are computed from instance_type when left unset. (see below for nested schema)
  • max_replicas (Number) Maximum number of nodes in the pool; the cluster-autoscaler ceiling.
  • max_unhealthy (String) How many of the pool's nodes may be unhealthy before remediation pauses, as a percentage (50%) or a bare integer (1). Drop to 0% once the pool is stable.
  • min_replicas (Number) Minimum number of nodes in the pool; the cluster-autoscaler floor.
  • node_startup_timeout (String) How long a machine may take to reach Ready before it is remediated (Go duration). Raise it for slow first boots.
  • resources (Attributes) Explicit CPU and memory per node, replacing instance_type sizing. Set both fields or neither — setting only one is rejected when the pool renders. (see below for nested schema)
  • roles (List of String) Node roles. Each role r labels the pool's nodes node-role.kubernetes.io/<r>; use ["ingress-nginx"] for a pool hosting the tenant ingress controller.
  • storage_class (String) StorageClass for the worker node system disks. Worker VMs live-migrate, so the class must serve ReadWriteMany volumes — prefer a replicated, DRBD-backed class. Changing a configured value replaces the pool: an existing volume is never migrated to another class.
  • talos (Attributes) Talos worker image configuration, kept in sync with the parent cluster's Talos settings. Every field follows the chart when unset. (see below for nested schema)
  • version (String) Kubernetes major.minor version the pool joins. Must match the parent cluster's version and satisfy the Talos/Kubernetes support matrix against talos.version.
  • wait_for_ready (Boolean) Block on create/update until the Ready condition is true and any server-generated outputs the resource exposes are readable.
  • wait_timeout (String) Maximum time to wait when wait_for_ready is set (Go duration, e.g. 10m). Readiness and outputs share this budget; outputs that never appear leave a warning and their attributes stay null until the next refresh.

Read-Only

  • chart_version (String) Deployed chart version (status.version).
  • id (String) Synthetic identifier in the form namespace/name.
  • ready (Boolean) Whether the application's Ready condition is true.
  • uid (String) Server-assigned object UID (metadata.uid). Stable across updates; changes on recreate.

Nested Schema for gpus

Required:

  • name (String) GPU resource name, such as nvidia.com/AD102GL_L40S.

Nested Schema for images

Optional:

  • kubectl (String) Image used by the pool's talos-reconcile Job.

Nested Schema for kubelet

Optional:

  • eviction_hard_memory (String) Hard memory eviction threshold, absolute (200Mi) or a percentage (7%).
  • eviction_soft_memory (String) Soft memory eviction threshold, absolute (1Gi) or a percentage (10%).
  • kube_reserved_cpu (String) CPU reserved for kubelet and the container runtime.
  • kube_reserved_memory (String) Memory reserved for kubelet and the container runtime.
  • system_reserved_cpu (String) CPU reserved for the host OS.
  • system_reserved_memory (String) Memory reserved for the host OS.

Nested Schema for resources

Optional:

  • cpu (String) CPU available to each node (quantity, e.g. 4). Requires memory.
  • memory (String) Memory available to each node (quantity, e.g. 8Gi). Requires cpu.

Nested Schema for talos

Optional:

  • image_factory_url (String) Base URL of the Talos Image Factory serving the worker OS disk image. Point it at a mirror or self-hosted factory for air-gapped installs. No trailing slash.
  • installer_repository (String) OCI repository prefix for the Talos installer image, resolved as <installer_repository>/<schematic_id>:<version>. No trailing slash.
  • schematic_id (String) Talos image-factory schematic ID. Set it for a custom schematic carrying system extensions or extra kernel arguments.
  • version (String) Talos release used for the worker OS image and installer.

Import

Import is supported using the following syntax:

The terraform import command can be used, for example:

# Kubernetes worker node pools are imported as "<namespace>/<cluster>-<pool>".
terraform import cozystack_kubernetes_nodes.gpu tenant-root/demo-gpu