Skip to content

Commit

Permalink
feat: Node pools can enable fast_socket (#2200)
Browse files Browse the repository at this point in the history
  • Loading branch information
@fraenkel
fraenkel authored Dec 7, 2024
1 parent 1f85f66 commit f23d52f
Show file tree
Hide file tree
Showing 16 changed files with 154 additions and 0 deletions.
1 change: 1 addition & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,7 @@ The node_pools variable takes the following parameters:
| disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
| disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
| effect | Effect for the taint | | Required |
| enable_fast_socket | Enable the NCCL Fast Socket feature. `enable_gvnic` must also be enabled. | null | Optional |
| enable_gcfs | Google Container File System (gcfs) has to be enabled for image streaming to be active. Needs image_type to be set to COS_CONTAINERD. | false | Optional |
| enable_gvnic | gVNIC (GVE) is an alternative to the virtIO-based ethernet driver. Needs a Container-Optimized OS node image. | false | Optional |
| enable_integrity_monitoring | Enables monitoring and attestation of the boot integrity of the instance. The attestation is performed against the integrity policy baseline. This baseline is initially derived from the implicitly trusted boot image when the instance is created. | true | Optional |
Expand Down
1 change: 1 addition & 0 deletions autogen/main/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,7 @@ The node_pools variable takes the following parameters:
| disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
| disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
| effect | Effect for the taint | | Required |
| enable_fast_socket | Enable the NCCL Fast Socket feature. `enable_gvnic` must also be enabled. | null | Optional |
| enable_gcfs | Google Container File System (gcfs) has to be enabled for image streaming to be active. Needs image_type to be set to COS_CONTAINERD. | false | Optional |
| enable_gvnic | gVNIC (GVE) is an alternative to the virtIO-based ethernet driver. Needs a Container-Optimized OS node image. | false | Optional |
| enable_integrity_monitoring | Enables monitoring and attestation of the boot integrity of the instance. The attestation is performed against the integrity policy baseline. This baseline is initially derived from the implicitly trusted boot image when the instance is created. | true | Optional |
Expand Down
13 changes: 13 additions & 0 deletions autogen/main/cluster.tf.tmpl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -537,6 +537,13 @@ resource "google_container_cluster" "primary" {
}
}

dynamic "fast_socket" {
for_each = lookup(var.node_pools[0], "enable_fast_socket", null) != null ? [var.node_pools[0].enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}

dynamic "kubelet_config" {
for_each = length(setintersection(
keys(var.node_pools[0]),
Expand Down Expand Up @@ -930,6 +937,12 @@ resource "google_container_node_pool" "windows_pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down
19 changes: 19 additions & 0 deletions cluster.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -418,6 +418,13 @@ resource "google_container_cluster" "primary" {
}
}

dynamic "fast_socket" {
for_each = lookup(var.node_pools[0], "enable_fast_socket", null) != null ? [var.node_pools[0].enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}

dynamic "kubelet_config" {
for_each = length(setintersection(
keys(var.node_pools[0]),
Expand Down Expand Up @@ -641,6 +648,12 @@ resource "google_container_node_pool" "pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down Expand Up @@ -932,6 +945,12 @@ resource "google_container_node_pool" "windows_pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down
1 change: 1 addition & 0 deletions modules/beta-private-cluster-update-variant/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -371,6 +371,7 @@ The node_pools variable takes the following parameters:
| disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
| disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
| effect | Effect for the taint | | Required |
| enable_fast_socket | Enable the NCCL Fast Socket feature. `enable_gvnic` must also be enabled. | null | Optional |
| enable_gcfs | Google Container File System (gcfs) has to be enabled for image streaming to be active. Needs image_type to be set to COS_CONTAINERD. | false | Optional |
| enable_gvnic | gVNIC (GVE) is an alternative to the virtIO-based ethernet driver. Needs a Container-Optimized OS node image. | false | Optional |
| enable_integrity_monitoring | Enables monitoring and attestation of the boot integrity of the instance. The attestation is performed against the integrity policy baseline. This baseline is initially derived from the implicitly trusted boot image when the instance is created. | true | Optional |
Expand Down
19 changes: 19 additions & 0 deletions modules/beta-private-cluster-update-variant/cluster.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -456,6 +456,13 @@ resource "google_container_cluster" "primary" {
}
}

dynamic "fast_socket" {
for_each = lookup(var.node_pools[0], "enable_fast_socket", null) != null ? [var.node_pools[0].enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}

dynamic "kubelet_config" {
for_each = length(setintersection(
keys(var.node_pools[0]),
Expand Down Expand Up @@ -799,6 +806,12 @@ resource "google_container_node_pool" "pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down Expand Up @@ -1104,6 +1117,12 @@ resource "google_container_node_pool" "windows_pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down
1 change: 1 addition & 0 deletions modules/beta-private-cluster/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,7 @@ The node_pools variable takes the following parameters:
| disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
| disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
| effect | Effect for the taint | | Required |
| enable_fast_socket | Enable the NCCL Fast Socket feature. `enable_gvnic` must also be enabled. | null | Optional |
| enable_gcfs | Google Container File System (gcfs) has to be enabled for image streaming to be active. Needs image_type to be set to COS_CONTAINERD. | false | Optional |
| enable_gvnic | gVNIC (GVE) is an alternative to the virtIO-based ethernet driver. Needs a Container-Optimized OS node image. | false | Optional |
| enable_integrity_monitoring | Enables monitoring and attestation of the boot integrity of the instance. The attestation is performed against the integrity policy baseline. This baseline is initially derived from the implicitly trusted boot image when the instance is created. | true | Optional |
Expand Down
19 changes: 19 additions & 0 deletions modules/beta-private-cluster/cluster.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -456,6 +456,13 @@ resource "google_container_cluster" "primary" {
}
}

dynamic "fast_socket" {
for_each = lookup(var.node_pools[0], "enable_fast_socket", null) != null ? [var.node_pools[0].enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}

dynamic "kubelet_config" {
for_each = length(setintersection(
keys(var.node_pools[0]),
Expand Down Expand Up @@ -718,6 +725,12 @@ resource "google_container_node_pool" "pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down Expand Up @@ -1022,6 +1035,12 @@ resource "google_container_node_pool" "windows_pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down
1 change: 1 addition & 0 deletions modules/beta-public-cluster-update-variant/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,7 @@ The node_pools variable takes the following parameters:
| disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
| disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
| effect | Effect for the taint | | Required |
| enable_fast_socket | Enable the NCCL Fast Socket feature. `enable_gvnic` must also be enabled. | null | Optional |
| enable_gcfs | Google Container File System (gcfs) has to be enabled for image streaming to be active. Needs image_type to be set to COS_CONTAINERD. | false | Optional |
| enable_gvnic | gVNIC (GVE) is an alternative to the virtIO-based ethernet driver. Needs a Container-Optimized OS node image. | false | Optional |
| enable_integrity_monitoring | Enables monitoring and attestation of the boot integrity of the instance. The attestation is performed against the integrity policy baseline. This baseline is initially derived from the implicitly trusted boot image when the instance is created. | true | Optional |
Expand Down
19 changes: 19 additions & 0 deletions modules/beta-public-cluster-update-variant/cluster.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -456,6 +456,13 @@ resource "google_container_cluster" "primary" {
}
}

dynamic "fast_socket" {
for_each = lookup(var.node_pools[0], "enable_fast_socket", null) != null ? [var.node_pools[0].enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}

dynamic "kubelet_config" {
for_each = length(setintersection(
keys(var.node_pools[0]),
Expand Down Expand Up @@ -769,6 +776,12 @@ resource "google_container_node_pool" "pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down Expand Up @@ -1074,6 +1087,12 @@ resource "google_container_node_pool" "windows_pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down
1 change: 1 addition & 0 deletions modules/beta-public-cluster/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,7 @@ The node_pools variable takes the following parameters:
| disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
| disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
| effect | Effect for the taint | | Required |
| enable_fast_socket | Enable the NCCL Fast Socket feature. `enable_gvnic` must also be enabled. | null | Optional |
| enable_gcfs | Google Container File System (gcfs) has to be enabled for image streaming to be active. Needs image_type to be set to COS_CONTAINERD. | false | Optional |
| enable_gvnic | gVNIC (GVE) is an alternative to the virtIO-based ethernet driver. Needs a Container-Optimized OS node image. | false | Optional |
| enable_integrity_monitoring | Enables monitoring and attestation of the boot integrity of the instance. The attestation is performed against the integrity policy baseline. This baseline is initially derived from the implicitly trusted boot image when the instance is created. | true | Optional |
Expand Down
19 changes: 19 additions & 0 deletions modules/beta-public-cluster/cluster.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -456,6 +456,13 @@ resource "google_container_cluster" "primary" {
}
}

dynamic "fast_socket" {
for_each = lookup(var.node_pools[0], "enable_fast_socket", null) != null ? [var.node_pools[0].enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}

dynamic "kubelet_config" {
for_each = length(setintersection(
keys(var.node_pools[0]),
Expand Down Expand Up @@ -688,6 +695,12 @@ resource "google_container_node_pool" "pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down Expand Up @@ -992,6 +1005,12 @@ resource "google_container_node_pool" "windows_pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down
1 change: 1 addition & 0 deletions modules/private-cluster-update-variant/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -353,6 +353,7 @@ The node_pools variable takes the following parameters:
| disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
| disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
| effect | Effect for the taint | | Required |
| enable_fast_socket | Enable the NCCL Fast Socket feature. `enable_gvnic` must also be enabled. | null | Optional |
| enable_gcfs | Google Container File System (gcfs) has to be enabled for image streaming to be active. Needs image_type to be set to COS_CONTAINERD. | false | Optional |
| enable_gvnic | gVNIC (GVE) is an alternative to the virtIO-based ethernet driver. Needs a Container-Optimized OS node image. | false | Optional |
| enable_integrity_monitoring | Enables monitoring and attestation of the boot integrity of the instance. The attestation is performed against the integrity policy baseline. This baseline is initially derived from the implicitly trusted boot image when the instance is created. | true | Optional |
Expand Down
19 changes: 19 additions & 0 deletions modules/private-cluster-update-variant/cluster.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -418,6 +418,13 @@ resource "google_container_cluster" "primary" {
}
}

dynamic "fast_socket" {
for_each = lookup(var.node_pools[0], "enable_fast_socket", null) != null ? [var.node_pools[0].enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}

dynamic "kubelet_config" {
for_each = length(setintersection(
keys(var.node_pools[0]),
Expand Down Expand Up @@ -751,6 +758,12 @@ resource "google_container_node_pool" "pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down Expand Up @@ -1043,6 +1056,12 @@ resource "google_container_node_pool" "windows_pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down
1 change: 1 addition & 0 deletions modules/private-cluster/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -331,6 +331,7 @@ The node_pools variable takes the following parameters:
| disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
| disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
| effect | Effect for the taint | | Required |
| enable_fast_socket | Enable the NCCL Fast Socket feature. `enable_gvnic` must also be enabled. | null | Optional |
| enable_gcfs | Google Container File System (gcfs) has to be enabled for image streaming to be active. Needs image_type to be set to COS_CONTAINERD. | false | Optional |
| enable_gvnic | gVNIC (GVE) is an alternative to the virtIO-based ethernet driver. Needs a Container-Optimized OS node image. | false | Optional |
| enable_integrity_monitoring | Enables monitoring and attestation of the boot integrity of the instance. The attestation is performed against the integrity policy baseline. This baseline is initially derived from the implicitly trusted boot image when the instance is created. | true | Optional |
Expand Down
19 changes: 19 additions & 0 deletions modules/private-cluster/cluster.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -418,6 +418,13 @@ resource "google_container_cluster" "primary" {
}
}

dynamic "fast_socket" {
for_each = lookup(var.node_pools[0], "enable_fast_socket", null) != null ? [var.node_pools[0].enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}

dynamic "kubelet_config" {
for_each = length(setintersection(
keys(var.node_pools[0]),
Expand Down Expand Up @@ -671,6 +678,12 @@ resource "google_container_node_pool" "pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down Expand Up @@ -962,6 +975,12 @@ resource "google_container_node_pool" "windows_pools" {
enabled = gvnic.value
}
}
dynamic "fast_socket" {
for_each = lookup(each.value, "enable_fast_socket", null) != null ? [each.value.enable_fast_socket] : []
content {
enabled = fast_socket.value
}
}
dynamic "reservation_affinity" {
for_each = lookup(each.value, "queued_provisioning", false) || lookup(each.value, "consume_reservation_type", "") != "" ? [each.value] : []
content {
Expand Down

0 comments on commit f23d52f

Please sign in to comment.