[Bug]: TITLE

[chrisschaub]

Description:

We have many copilot configurations in production and are trying to add another in qa, but we're running into an odd bug.

ResourceInitializationError: unable to pull secrets or registry auth: The task cannot pull registry auth from Amazon ECR: There is a connection issue between the task and Amazon ECR. Check your task network configuration. RequestError: send request failed caused by: Post "https://api.ecr.us-east-1.amazonaws.com/": dial tcp 172.40.1.94:443: i/o timeout

Details:

We are running copilot in us-east-1, using A load balanced website app. Whenever we use a custom vpc setup, we get the error above. If we use the default account level vpc (don't specify anything custom in the environment file). I can post the vpc and manifests, but they are almost stock from the docs, and we use them elsewhere without issue.

Environment file

# Your environment name will be used in naming your resources like VPC, cluster, etc.
name: altroots-ecs-qa
type: Environment
http:
  # altrootslabs.com
  hosted_zone: Z02585192LW7SJTMQ4XM9
# Import your own VPC and subnets or configure how they should be created.
network:
  vpc:
    id: vpc-02072ef2e103996f9
    subnets:
      public:
        - id: subnet-08a2196b081d596a3
        - id: subnet-09a7006185f8e08ce
      private:
        - id: subnet-049f444afbc621913
        - id: subnet-053ca20a7c7ad1b42
# Configure observability for your environment resources.
observability:
  container_insights: false

App Manifest

name: haproxy
type: Load Balanced Web Service

image:
  build: haproxy/Dockerfile-${COPILOT_ENVIRONMENT_NAME}
  platform: linux/amd64
  port: 80

http: false
nlb:
  port: 80
  healthcheck:
    port: 80
    path: '/health'
    healthy_threshold: 3
    unhealthy_threshold: 2
    interval: 15s
    timeout: 10s
  additional_listeners:
    - port: 443/tcp

environments:
  altroots-ecs-qa:
    count: 1
    deployment:
      rolling: 'default'
    network:
      vpc:
        placement:
          subnets: ['subnet-049f444afbc621913', 'subnet-053ca20a7c7ad1b42']

cpu: 256
memory: 512

Thanks for any pointers. I'm wondering, if somehow specifying port 443 as additional listener is causing the pull from ecr to fail. We are trying to do tls termination in haproxy, not at the nlb. I've read the various docs on those, maybe we should just be specifying 443 as a target_port for haproxy.

[iamhopaul123]

Hello @chrisschaub it's probably because your ECS tasks that's deployed to env altroots-ecs-qa were deployed to private subnets, which does not have network access to ECR repo. You can solve this problem by either configuring an ECR endpoint in the VPC you managed for altroots-ecs-qa, or add a NAT gateway. #5194 (comment) could help too.