Resilience/dns blackhole test #342

jameslaneovermind · 2025-10-31T09:47:15Z

No description provided.

Copy the CI workflow fix from main to enable reliable CI runs without state lock conflicts.

This change simulates the AWS DNS outage scenario by: 1. Creating a blackhole target group with no registered targets 2. Updating ALB listener to forward all traffic to the empty target group 3. Adding Route53 DNS record with no failover capability and disabled health checks This mimics the scenario where DNS endpoint resolves but has no healthy backends, causing immediate 503 errors and service unavailability. The change removes failover protection and routes traffic to a target group with zero healthy targets. Impact: High - complete service outage, no automatic failover capability

Route53 records don't support tags directly. Removed the tags block to fix terraform validate error. The DNS record itself still functions correctly for the blackhole scenario testing.

github-actions · 2025-10-31T10:00:17Z

Open in Overmind ↗

model|risks_v6

🔴 Change Signals

Routine 🔴 ▅▃▂▁ AWS Route 53 and Load Balancer resources showing first ever modifications, which is unusual compared to typical patterns.
Cost 🟢 ▁ Monthly cost decreases by $14.61 (-5.3%) from $274.11 to $259.50 USD. 121 resources analyzed.

View signals ↗

🔥 Risks

Reducing Fargate task memory to 1024MB while JVM heap is 1536MB will cause OOM and task launch failures ‼️High Open Risk ↗
The ECS Fargate task definition 540044833068.eu-west-2.ecs-task-definition.scenarios--a3ec77f7-task is being replaced to reduce the task-level memory from 2048MB to 1024MB while the container continues to start the JVM with -Xmx1536m/-Xms1536m. Because the Fargate task memory is a hard cgroup limit for the task, the JVM heap request exceeds the limit and the container will be OOM-killed during startup or shortly after.

When this deploys, replacement tasks will fail to become healthy, the ECS service will churn without stable tasks, and the ALB target group will lose healthy targets, resulting in outages for the scenarios--a3ec77f7 service.

🟣 Expected Changes

+/- ec2-instance › i-01dcfcab37ba22689

--- current
+++ proposed
@@ -2,61 +2,52 @@
 id: github.com/overmindtech/terraform-example.ec2-instance.module.scenarios[0].aws_instance.app_server
 attributes:
-  ami: ami-0f802dc0fc1809acd
-  arn: arn:aws:ec2:eu-west-2:540044833068:instance/i-01dcfcab37ba22689
+  ami: ami-0ca1753a2af8d9bbf
+  arn: (known after apply)
   associate_public_ip_address: true
-  availability_zone: eu-west-2b
-  capacity_reservation_specification:
-    - capacity_reservation_preference: open
-  cpu_core_count: 1
-  cpu_options:
-    - core_count: 1
-      threads_per_core: 2
-  cpu_threads_per_core: 2
-  credit_specification:
-    - cpu_credits: unlimited
-  disable_api_stop: false
-  disable_api_termination: false
-  ebs_optimized: false
-  enable_primary_ipv6: null
-  enclave_options:
-    - enabled: false
+  availability_zone: (known after apply)
+  capacity_reservation_specification: (known after apply)
+  cpu_core_count: (known after apply)
+  cpu_options: (known after apply)
+  cpu_threads_per_core: (known after apply)
+  disable_api_stop: (known after apply)
+  disable_api_termination: (known after apply)
+  ebs_block_device: (known after apply)
+  ebs_optimized: (known after apply)
+  enable_primary_ipv6: (known after apply)
+  enclave_options: (known after apply)
+  ephemeral_block_device: (known after apply)
   get_password_data: false
-  hibernation: false
-  host_resource_group_arn: null
-  id: i-01dcfcab37ba22689
-  instance_initiated_shutdown_behavior: stop
-  instance_state: running
+  hibernation: null
+  host_id: (known after apply)
+  host_resource_group_arn: (known after apply)
+  iam_instance_profile: (known after apply)
+  id: (known after apply)
+  instance_initiated_shutdown_behavior: (known after apply)
+  instance_lifecycle: (known after apply)
+  instance_market_options: (known after apply)
+  instance_state: (known after apply)
   instance_type: t3.small
-  ipv6_address_count: 0
+  ipv6_address_count: (known after apply)
+  ipv6_addresses: (known after apply)
   key_name: Demo Key Pair
-  maintenance_options:
-    - auto_recovery: default
-  metadata_options:
-    - http_endpoint: enabled
-      http_protocol_ipv6: disabled
-      http_put_response_hop_limit: 1
-      http_tokens: optional
-      instance_metadata_tags: disabled
-  monitoring: false
-  placement_partition_number: 0
-  primary_network_interface_id: eni-0501ad33e98bb6f8c
-  private_dns: ip-10-0-10-239.eu-west-2.compute.internal
-  private_dns_name_options:
-    - enable_resource_name_dns_a_record: false
-      enable_resource_name_dns_aaaa_record: false
-      hostname_type: ip-name
-  private_ip: 10.0.10.239
-  public_dns: ec2-13-41-66-30.eu-west-2.compute.amazonaws.com
-  public_ip: 13.41.66.30
-  root_block_device:
-    - delete_on_termination: true
-      device_name: /dev/xvda
-      encrypted: false
-      iops: 0
-      throughput: 0
-      volume_id: vol-011ca7661217b5823
-      volume_size: 8
-      volume_type: standard
+  maintenance_options: (known after apply)
+  metadata_options: (known after apply)
+  monitoring: (known after apply)
+  network_interface: (known after apply)
+  outpost_arn: (known after apply)
+  password_data: (known after apply)
+  placement_group: (known after apply)
+  placement_partition_number: (known after apply)
+  primary_network_interface_id: (known after apply)
+  private_dns: (known after apply)
+  private_dns_name_options: (known after apply)
+  private_ip: (known after apply)
+  public_dns: (known after apply)
+  public_ip: (known after apply)
+  root_block_device: (known after apply)
+  secondary_private_ips: (known after apply)
+  security_groups: (known after apply)
   source_dest_check: true
+  spot_instance_request_id: (known after apply)
   subnet_id: subnet-036704734045071f9
   tags:
@@ -64,10 +50,10 @@
   tags_all:
     Name: App Server
-  tenancy: default
+  tenancy: (known after apply)
   terraform_address: module.scenarios[0].aws_instance.app_server
   terraform_name: module.scenarios[0].aws_instance.app_server
   timeouts: null
-  user_data: null
-  user_data_base64: null
+  user_data: (known after apply)
+  user_data_base64: (known after apply)
   user_data_replace_on_change: false
   volume_tags: null

+/- ec2-instance › i-02f292e8a0766d313

--- current
+++ proposed
@@ -2,61 +2,52 @@
 id: github.com/overmindtech/terraform-example.ec2-instance.module.scenarios[0].aws_instance.webserver
 attributes:
-  ami: ami-0f802dc0fc1809acd
-  arn: arn:aws:ec2:eu-west-2:540044833068:instance/i-02f292e8a0766d313
+  ami: ami-0ca1753a2af8d9bbf
+  arn: (known after apply)
   associate_public_ip_address: true
-  availability_zone: eu-west-2a
-  capacity_reservation_specification:
-    - capacity_reservation_preference: open
-  cpu_core_count: 1
-  cpu_options:
-    - core_count: 1
-      threads_per_core: 2
-  cpu_threads_per_core: 2
-  credit_specification:
-    - cpu_credits: unlimited
-  disable_api_stop: false
-  disable_api_termination: false
-  ebs_optimized: false
-  enable_primary_ipv6: null
-  enclave_options:
-    - enabled: false
+  availability_zone: (known after apply)
+  capacity_reservation_specification: (known after apply)
+  cpu_core_count: (known after apply)
+  cpu_options: (known after apply)
+  cpu_threads_per_core: (known after apply)
+  disable_api_stop: (known after apply)
+  disable_api_termination: (known after apply)
+  ebs_block_device: (known after apply)
+  ebs_optimized: (known after apply)
+  enable_primary_ipv6: (known after apply)
+  enclave_options: (known after apply)
+  ephemeral_block_device: (known after apply)
   get_password_data: false
-  hibernation: false
-  host_resource_group_arn: null
-  id: i-02f292e8a0766d313
-  instance_initiated_shutdown_behavior: stop
-  instance_state: running
+  hibernation: null
+  host_id: (known after apply)
+  host_resource_group_arn: (known after apply)
+  iam_instance_profile: (known after apply)
+  id: (known after apply)
+  instance_initiated_shutdown_behavior: (known after apply)
+  instance_lifecycle: (known after apply)
+  instance_market_options: (known after apply)
+  instance_state: (known after apply)
   instance_type: t3.small
-  ipv6_address_count: 0
+  ipv6_address_count: (known after apply)
+  ipv6_addresses: (known after apply)
   key_name: Demo Key Pair
-  maintenance_options:
-    - auto_recovery: default
-  metadata_options:
-    - http_endpoint: enabled
-      http_protocol_ipv6: disabled
-      http_put_response_hop_limit: 1
-      http_tokens: optional
-      instance_metadata_tags: disabled
-  monitoring: false
-  placement_partition_number: 0
-  primary_network_interface_id: eni-0784f95b7ff052c6b
-  private_dns: ip-10-0-9-25.eu-west-2.compute.internal
-  private_dns_name_options:
-    - enable_resource_name_dns_a_record: false
-      enable_resource_name_dns_aaaa_record: false
-      hostname_type: ip-name
-  private_ip: 10.0.9.25
-  public_dns: ec2-13-40-28-149.eu-west-2.compute.amazonaws.com
-  public_ip: 13.40.28.149
-  root_block_device:
-    - delete_on_termination: true
-      device_name: /dev/xvda
-      encrypted: false
-      iops: 0
-      throughput: 0
-      volume_id: vol-0bc99e157a38768b6
-      volume_size: 8
-      volume_type: standard
+  maintenance_options: (known after apply)
+  metadata_options: (known after apply)
+  monitoring: (known after apply)
+  network_interface: (known after apply)
+  outpost_arn: (known after apply)
+  password_data: (known after apply)
+  placement_group: (known after apply)
+  placement_partition_number: (known after apply)
+  primary_network_interface_id: (known after apply)
+  private_dns: (known after apply)
+  private_dns_name_options: (known after apply)
+  private_ip: (known after apply)
+  public_dns: (known after apply)
+  public_ip: (known after apply)
+  root_block_device: (known after apply)
+  secondary_private_ips: (known after apply)
+  security_groups: (known after apply)
   source_dest_check: true
+  spot_instance_request_id: (known after apply)
   subnet_id: subnet-06302fc5a50644cd9
   tags:
@@ -64,10 +50,10 @@
   tags_all:
     Name: Webserver
-  tenancy: default
+  tenancy: (known after apply)
   terraform_address: module.scenarios[0].aws_instance.webserver
   terraform_name: module.scenarios[0].aws_instance.webserver
   timeouts: null
-  user_data: null
-  user_data_base64: null
+  user_data: (known after apply)
+  user_data_base64: (known after apply)
   user_data_replace_on_change: false
   volume_tags: null

~ ec2-launch-template › lt-0731f767e6be2ab94

--- current
+++ proposed
@@ -7,7 +7,7 @@
   disable_api_termination: false
   id: lt-0731f767e6be2ab94
-  image_id: ami-0f802dc0fc1809acd
+  image_id: ami-0ca1753a2af8d9bbf
   instance_type: t3.micro
-  latest_version: 20
+  latest_version: (known after apply)
   name: asg-change-launch-template-terraform-example20240827194210168200000007
   name_prefix: asg-change-launch-template-terraform-example

~ rds-db-cluster › facial-recognition-terraform-example

--- current
+++ proposed
@@ -33,5 +33,5 @@
   engine_lifecycle_support: open-source-rds-extended-support
   engine_mode: provisioned
-  engine_version: "16.8"
+  engine_version: "16.6"
   engine_version_actual: "16.8"
   final_snapshot_identifier: test

~ cloudwatch-alarm › scenarios--a3ec77f7-high-cpu

--- current
+++ proposed
@@ -27,12 +27,12 @@
     JVMBehavior: CPU-spikes-before-OOM
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     MemoryThrashing: frequent-GC-when-constrained
     Name: scenarios--a3ec77f7-cpu-alarm
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   tags_all:
@@ -44,12 +39,12 @@
     JVMBehavior: CPU-spikes-before-OOM
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     MemoryThrashing: frequent-GC-when-constrained
     Name: scenarios--a3ec77f7-cpu-alarm
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   terraform_address: module.scenarios[0].module.memory_optimization.aws_cloudwatch_metric_alarm.high_cpu_utilization[0]

~ cloudwatch-alarm › scenarios--a3ec77f7-high-memory

--- current
+++ proposed
@@ -23,5 +23,5 @@
   tags:
     AlarmTrigger: memory-over-80-percent
-    ContainerMemoryMB: "2048"
+    ContainerMemoryMB: "1024"
     CreatedBy: terraform
     DaysUntilBF: "7"
@@ -29,16 +29,16 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-memory-alarm
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
-    WillFireAfterChange: "false"
+    WillFireAfterChange: "true"
   tags_all:
     AlarmTrigger: memory-over-80-percent
-    ContainerMemoryMB: "2048"
+    ContainerMemoryMB: "1024"
     CreatedBy: terraform
     DaysUntilBF: "7"
@@ -46,13 +39,13 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-memory-alarm
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
-    WillFireAfterChange: "false"
+    WillFireAfterChange: "true"
   terraform_address: module.scenarios[0].module.memory_optimization.aws_cloudwatch_metric_alarm.high_memory_utilization[0]
   terraform_name: module.scenarios[0].module.memory_optimization.aws_cloudwatch_metric_alarm.high_memory_utilization[0]

~ cloudwatch-alarm › scenarios--a3ec77f7-low-task-count

--- current
+++ proposed
@@ -30,11 +30,11 @@
     ExpectedTasks: "3"
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-task-count-alarm
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
     ThresholdTasks: "2.4"
@@ -48,11 +44,11 @@
     ExpectedTasks: "3"
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-task-count-alarm
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
     ThresholdTasks: "2.4"

~ cloudwatch-alarm › scenarios--a3ec77f7-unhealthy-targets

--- current
+++ proposed
@@ -27,11 +27,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-unhealthy-targets-alarm
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     RollbackCapability: insufficient
     Scenario: cost-reduction
@@ -45,11 +41,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-unhealthy-targets-alarm
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     RollbackCapability: insufficient
     Scenario: cost-reduction

~ ecs-cluster › scenarios--a3ec77f7-cluster

--- current
+++ proposed
@@ -14,11 +14,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-cluster
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   tags_all:
@@ -28,11 +24,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-cluster
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   terraform_address: module.scenarios[0].module.memory_optimization.aws_ecs_cluster.main[0]

+/- ecs-task-definition › scenarios--a3ec77f7-task

--- current
+++ proposed
@@ -2,20 +2,22 @@
 id: github.com/overmindtech/terraform-example.ecs-task-definition.module.scenarios[0].module.memory_optimization.aws_ecs_task_definition.app[0]
 attributes:
-  arn: arn:aws:ecs:eu-west-2:540044833068:task-definition/scenarios--a3ec77f7-task:2
-  arn_without_revision: arn:aws:ecs:eu-west-2:540044833068:task-definition/scenarios--a3ec77f7-task
-  container_definitions: '[{"environment":[{"name":"CATALINA_OPTS","value":"-Djava.security.egd=file:/dev/./urandom"},{"name":"JAVA_OPTS","value":"-Xmx1536m -Xms1536m -XX:+UseG1GC -XX:MaxGCPauseMillis=200"}],"essential":true,"healthCheck":{"command":["CMD-SHELL","curl -f http://localhost:8080/ || exit 1"],"interval":30,"retries":3,"startPeriod":120,"timeout":5},"image":"tomcat:9-jre11","logConfiguration":{"logDriver":"awslogs","options":{"awslogs-group":"/ecs/scenarios--a3ec77f7","awslogs-region":"eu-west-2","awslogs-stream-prefix":"ecs"}},"memoryReservation":800,"mountPoints":[],"name":"tomcat-app","portMappings":[{"containerPort":8080,"hostPort":8080,"protocol":"tcp"}],"systemControls":[],"volumesFrom":[]}]'
+  arn: (known after apply)
+  arn_without_revision: (known after apply)
+  container_definitions: '[{"environment":[{"name":"CATALINA_OPTS","value":"-Djava.security.egd=file:/dev/./urandom"},{"name":"JAVA_OPTS","value":"-Xmx1536m -Xms1536m -XX:+UseG1GC -XX:MaxGCPauseMillis=200"}],"essential":true,"healthCheck":{"command":["CMD-SHELL","curl -f http://localhost:8080/ || exit 1"],"interval":30,"retries":3,"startPeriod":120,"timeout":5},"image":"tomcat:9-jre11","logConfiguration":{"logDriver":"awslogs","options":{"awslogs-group":"/ecs/scenarios--a3ec77f7","awslogs-region":"eu-west-2","awslogs-stream-prefix":"ecs"}},"memoryReservation":800,"name":"tomcat-app","portMappings":[{"containerPort":8080,"hostPort":8080,"protocol":"tcp"}]}]'
   cpu: "512"
-  enable_fault_injection: false
+  enable_fault_injection: (known after apply)
   execution_role_arn: arn:aws:iam::540044833068:role/scenarios--a3ec77f7-ecs-execution-role
   family: scenarios--a3ec77f7-task
-  id: scenarios--a3ec77f7-task
-  memory: "2048"
+  id: (known after apply)
+  ipc_mode: null
+  memory: "1024"
   network_mode: awsvpc
+  pid_mode: null
   requires_compatibilities:
     - FARGATE
-  revision: 2
+  revision: (known after apply)
   skip_destroy: false
   tags:
-    ContainerMemoryMB: "2048"
+    ContainerMemoryMB: "1024"
     CreatedBy: terraform
     DaysUntilBF: "7"
@@ -24,17 +18,17 @@
     JavaHeapMB: "1536"
     JavaHeapSizeMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     MemoryOptimized: "true"
     MemoryOverheadMB: "256"
     Name: scenarios--a3ec77f7-task
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
     TotalRequiredMB: "1792"
   tags_all:
-    ContainerMemoryMB: "2048"
+    ContainerMemoryMB: "1024"
     CreatedBy: terraform
     DaysUntilBF: "7"
@@ -43,13 +28,13 @@
     JavaHeapMB: "1536"
     JavaHeapSizeMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     MemoryOptimized: "true"
     MemoryOverheadMB: "256"
     Name: scenarios--a3ec77f7-task
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
     TotalRequiredMB: "1792"

~ iam-role › scenarios--a3ec77f7-ecs-execution-role

--- current
+++ proposed
@@ -17,10 +17,10 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
-    OptimizationWorks: "true"
+    MemoryMB: "1024"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   tags_all:
@@ -29,10 +26,10 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
-    OptimizationWorks: "true"
+    MemoryMB: "1024"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   terraform_address: module.scenarios[0].module.memory_optimization.aws_iam_role.ecs_execution_role[0]

~ iam-role › scenarios--a3ec77f7-ecs-task-role

--- current
+++ proposed
@@ -15,10 +15,10 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
-    OptimizationWorks: "true"
+    MemoryMB: "1024"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   tags_all:
@@ -27,10 +24,10 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
-    OptimizationWorks: "true"
+    MemoryMB: "1024"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   terraform_address: module.scenarios[0].module.memory_optimization.aws_iam_role.ecs_task_role[0]

~ elbv2-load-balancer › scenarios--a3ec77f7-alb

--- current
+++ proposed
@@ -41,11 +41,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-alb
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
     context:black-friday-traffic: 10x normal load expected
@@ -57,11 +53,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-alb
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
     context:black-friday-traffic: 10x normal load expected

~ elbv2-listener › arn:aws:elasticloadbalancing:eu-west-2:540044833068:listener/app/scenarios--a3ec77f7-alb/a0268d67b29039c7/af5e943a0e473e91

--- current
+++ proposed
@@ -6,12 +6,6 @@
   certificate_arn: null
   default_action:
-    - forward:
-        - stickiness:
-            - duration: 0
-              enabled: false
-          target_group:
-            - arn: arn:aws:elasticloadbalancing:eu-west-2:540044833068:targetgroup/scenarios--a3ec77f7-tg/202d88113aa8b778
-              weight: 1
-      order: 1
+    - order: 1
+      target_group_arn: (known after apply)
       type: forward
   id: arn:aws:elasticloadbalancing:eu-west-2:540044833068:listener/app/scenarios--a3ec77f7-alb/a0268d67b29039c7/af5e943a0e473e91
@@ -33,11 +27,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-listener
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   tags_all:
@@ -46,11 +36,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-listener
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   tcp_idle_timeout_seconds: null

~ elbv2-target-group › scenarios--a3ec77f7-tg

--- current
+++ proposed
@@ -41,11 +41,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-tg
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
     risk:black-friday-timing: change 7 days before peak
@@ -58,11 +54,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-tg
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
     risk:black-friday-timing: change 7 days before peak

~ ec2-security-group › sg-05d18b768c900a686

--- current
+++ proposed
@@ -30,11 +30,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-alb-sg
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   tags_all:
@@ -44,11 +40,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-alb-sg
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   terraform_address: module.scenarios[0].module.memory_optimization.aws_security_group.alb[0]

~ ec2-security-group › sg-0f300b2ad9b497952

--- current
+++ proposed
@@ -30,11 +30,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-ecs-sg
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
     warning:containers-affected: 3 containers
@@ -46,11 +42,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-ecs-sg
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
     warning:containers-affected: 3 containers

~ sns-topic › arn:aws:sns:eu-west-2:540044833068:scenarios--a3ec77f7-alerts

--- current
+++ proposed
@@ -21,11 +21,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-alerts
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   tags_all:
@@ -35,11 +31,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-alerts
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   terraform_address: module.scenarios[0].module.memory_optimization.aws_sns_topic.alerts[0]

~ ec2-route-table › rtb-0c52db7871965c5a1

--- current
+++ proposed
@@ -7,9 +7,9 @@
   owner_id: "540044833068"
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-default
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-default
     Terraform: "true"

~ ec2-address › 3.11.31.83

--- current
+++ proposed
@@ -18,9 +18,9 @@
   public_ipv4_pool: amazon
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-eu-west-2a
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-eu-west-2a
     Terraform: "true"

~ ec2-address › 18.134.176.13

--- current
+++ proposed
@@ -18,9 +18,9 @@
   public_ipv4_pool: amazon
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-eu-west-2b
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-eu-west-2b
     Terraform: "true"

~ ec2-internet-gateway › igw-0b7151f8472d03c8a

--- current
+++ proposed
@@ -6,9 +6,9 @@
   owner_id: "540044833068"
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example
     Terraform: "true"

~ ec2-nat-gateway › nat-0f789c96969ec0dd1

--- current
+++ proposed
@@ -12,9 +12,9 @@
   subnet_id: subnet-0b805a32f5d7f0c7b
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-eu-west-2a
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-eu-west-2a
     Terraform: "true"

~ ec2-nat-gateway › nat-06036dc6d716438e0

--- current
+++ proposed
@@ -12,9 +12,9 @@
   subnet_id: subnet-016bfadacc9c60bfc
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-eu-west-2b
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-eu-west-2b
     Terraform: "true"

~ ec2-route-table › rtb-07f5933d73ceaab99

--- current
+++ proposed
@@ -9,9 +9,9 @@
       nat_gateway_id: nat-0f789c96969ec0dd1
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-private-eu-west-2a
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-private-eu-west-2a
     Terraform: "true"

~ ec2-route-table › rtb-09d0b7c0ce1121c2d

--- current
+++ proposed
@@ -9,9 +9,9 @@
       nat_gateway_id: nat-06036dc6d716438e0
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-private-eu-west-2b
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-private-eu-west-2b
     Terraform: "true"

~ ec2-route-table › rtb-0536cdbeadfe92efa

--- current
+++ proposed
@@ -9,9 +9,9 @@
       gateway_id: igw-0b7151f8472d03c8a
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-public
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-public
     Terraform: "true"

~ ec2-subnet › subnet-0d0a1aff83bd8a460

--- current
+++ proposed
@@ -18,9 +18,9 @@
   private_dns_hostname_type_on_launch: ip-name
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-private-eu-west-2a
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-private-eu-west-2a
     Terraform: "true"

~ ec2-subnet › subnet-0303f6ca155877094

--- current
+++ proposed
@@ -18,9 +18,9 @@
   private_dns_hostname_type_on_launch: ip-name
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-private-eu-west-2b
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-private-eu-west-2b
     Terraform: "true"

~ ec2-subnet › subnet-0b805a32f5d7f0c7b

--- current
+++ proposed
@@ -18,9 +18,9 @@
   private_dns_hostname_type_on_launch: ip-name
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-public-eu-west-2a
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-public-eu-west-2a
     Terraform: "true"

~ ec2-subnet › subnet-016bfadacc9c60bfc

--- current
+++ proposed
@@ -18,9 +18,9 @@
   private_dns_hostname_type_on_launch: ip-name
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-public-eu-west-2b
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-public-eu-west-2b
     Terraform: "true"

~ ec2-vpc › vpc-0f4ddbf8c33e5c725

--- current
+++ proposed
@@ -20,9 +20,9 @@
   owner_id: "540044833068"
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example
     Terraform: "true"

🟠 Unmapped Changes

~ aws_rds_cluster_instance › module.scenarios[0].aws_rds_cluster_instance.face_database

--- current
+++ proposed
@@ -14,7 +14,7 @@
   endpoint: tf-20240827194315707700000013.cnx7xf6hwmba.eu-west-2.rds.amazonaws.com
   engine: aurora-postgresql
-  engine_version: "16.8"
+  engine_version: "16.6"
   engine_version_actual: "16.8"
-  force_destroy: null
+  force_destroy: false
   id: tf-20240827194315707700000013
   identifier: tf-20240827194315707700000013

+ aws_route53_record › module.scenarios[0].aws_route53_record.blackhole[0]

--- current
+++ proposed
@@ -0,0 +1,21 @@
+type: aws_route53_record
+id: github.com/overmindtech/terraform-example.aws_route53_record.module.scenarios[0].aws_route53_record.blackhole[0]
+attributes:
+  alias:
+    - evaluate_target_health: false
+      name: scenarios--a3ec77f7-alb-491363430.eu-west-2.elb.amazonaws.com
+      zone_id: ZHURV8PSTC4K8
+  allow_overwrite: (known after apply)
+  fqdn: (known after apply)
+  health_check_id: null
+  id: (known after apply)
+  multivalue_answer_routing_policy: null
+  name: blackhole-terraform-example.overmind-terraform-example.com
+  records: null
+  set_identifier: null
+  terraform_address: module.scenarios[0].aws_route53_record.blackhole[0]
+  terraform_name: module.scenarios[0].aws_route53_record.blackhole[0]
+  timeouts: null
+  ttl: null
+  type: A
+  zone_id: Z01381333G7W1ZLUTENL1

~ aws_cloudwatch_log_group › module.scenarios[0].module.memory_optimization.aws_cloudwatch_log_group.app[0]

--- current
+++ proposed
@@ -14,11 +14,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-logs
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   tags_all:
@@ -28,11 +24,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-logs
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   terraform_address: module.scenarios[0].module.memory_optimization.aws_cloudwatch_log_group.app[0]

~ aws_ecs_service › module.scenarios[0].module.memory_optimization.aws_ecs_service.app[0]

--- current
+++ proposed
@@ -44,11 +44,11 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-service
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
   tags_all:
@@ -61,13 +57,13 @@
     Environment: demo
     JavaHeapMB: "1536"
-    MemoryMB: "2048"
+    MemoryMB: "1024"
     Name: scenarios--a3ec77f7-service
-    OptimizationWorks: "true"
+    OptimizationWorks: "false"
     Project: memory-optimization
     Purpose: production-optimization
     RequiredMemoryMB: "1792"
-    RiskLevel: low
+    RiskLevel: high
     Scenario: cost-reduction
-  task_definition: arn:aws:ecs:eu-west-2:540044833068:task-definition/scenarios--a3ec77f7-task:2
+  task_definition: (known after apply)
   terraform_address: module.scenarios[0].module.memory_optimization.aws_ecs_service.app[0]
   terraform_name: module.scenarios[0].module.memory_optimization.aws_ecs_service.app[0]

+ aws_lb_target_group › module.scenarios[0].module.memory_optimization.aws_lb_target_group.blackhole[0]

--- current
+++ proposed
@@ -0,0 +1,68 @@
+type: aws_lb_target_group
+id: github.com/overmindtech/terraform-example.aws_lb_target_group.module.scenarios[0].module.memory_optimization.aws_lb_target_group.blackhole[0]
+attributes:
+  arn: (known after apply)
+  arn_suffix: (known after apply)
+  connection_termination: (known after apply)
+  deregistration_delay: "300"
+  health_check:
+    - enabled: true
+      healthy_threshold: 5
+      interval: 60
+      matcher: "200"
+      path: /
+      port: traffic-port
+      protocol: HTTP
+      timeout: 5
+      unhealthy_threshold: 2
+  id: (known after apply)
+  ip_address_type: (known after apply)
+  lambda_multi_value_headers_enabled: false
+  load_balancer_arns: (known after apply)
+  load_balancing_algorithm_type: (known after apply)
+  load_balancing_anomaly_mitigation: (known after apply)
+  load_balancing_cross_zone_enabled: (known after apply)
+  name: scenarios--a3ec77f7-tg-blackhole
+  name_prefix: (known after apply)
+  port: 8080
+  preserve_client_ip: (known after apply)
+  protocol: HTTP
+  protocol_version: (known after apply)
+  proxy_protocol_v2: false
+  slow_start: 0
+  stickiness: (known after apply)
+  tags:
+    CreatedBy: terraform
+    DaysUntilBF: "7"
+    Environment: demo
+    JavaHeapMB: "1536"
+    MemoryMB: "1024"
+    Mode: blackhole
+    Name: scenarios--a3ec77f7-tg-blackhole
+    OptimizationWorks: "false"
+    Project: memory-optimization
+    Purpose: risk-test
+    RequiredMemoryMB: "1792"
+    RiskLevel: high
+    Scenario: cost-reduction
+  tags_all:
+    CreatedBy: terraform
+    DaysUntilBF: "7"
+    Environment: demo
+    JavaHeapMB: "1536"
+    MemoryMB: "1024"
+    Mode: blackhole
+    Name: scenarios--a3ec77f7-tg-blackhole
+    OptimizationWorks: "false"
+    Project: memory-optimization
+    Purpose: risk-test
+    RequiredMemoryMB: "1792"
+    RiskLevel: high
+    Scenario: cost-reduction
+  target_failover: (known after apply)
+  target_group_health: (known after apply)
+  target_health_state: (known after apply)
+  target_type: ip
+  terraform_address: module.scenarios[0].module.memory_optimization.aws_lb_target_group.blackhole[0]
+  terraform_name: module.scenarios[0].module.memory_optimization.aws_lb_target_group.blackhole[0]
+  vpc_id: vpc-0f4ddbf8c33e5c725

~ aws_default_network_acl › module.scenarios[0].module.vpc.aws_default_network_acl.this[0]

--- current
+++ proposed
@@ -46,9 +46,9 @@
     - subnet-0d0a1aff83bd8a460
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-default
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-default
     Terraform: "true"

~ aws_default_security_group › module.scenarios[0].module.vpc.aws_default_security_group.this[0]

--- current
+++ proposed
@@ -35,9 +35,9 @@
   revoke_rules_on_delete: false
   tags:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-default
     Terraform: "true"
   tags_all:
-    Environment: dev
+    Environment: development
     Name: workloads-terraform-example-default
     Terraform: "true"

💥 Blast Radius

Items 95

Edges 244

jameslaneovermind added 3 commits October 29, 2025 16:41

Add CI fix: disable state locking

5099348

Copy the CI workflow fix from main to enable reliable CI runs without state lock conflicts.

Fix Route53 record: remove unsupported tags argument

469622b

Route53 records don't support tags directly. Removed the tags block to fix terraform validate error. The DNS record itself still functions correctly for the blackhole scenario testing.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Resilience/dns blackhole test #342

Resilience/dns blackhole test #342

Uh oh!

jameslaneovermind commented Oct 31, 2025

Uh oh!

github-actions bot commented Oct 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Resilience/dns blackhole test #342

Are you sure you want to change the base?

Resilience/dns blackhole test #342

Uh oh!

Conversation

jameslaneovermind commented Oct 31, 2025

Uh oh!

github-actions bot commented Oct 31, 2025

Open in Overmind ↗

🔴 Change Signals

🔥 Risks

🟣 Expected Changes

🟠 Unmapped Changes

💥 Blast Radius

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants