Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: aws_cloudformation_stack: yaml: unmarshal errors, mapping key ... already defined #41252

Open
YakDriver opened this issue Feb 5, 2025 · 1 comment
Open

[Bug]: aws_cloudformation_stack: yaml: unmarshal errors, mapping key ... already defined #41252

YakDriver opened this issue Feb 5, 2025 · 1 comment
Labels
bug Addresses a defect in current functionality. service/cloudformation Issues and PRs that pertain to the cloudformation service.

Comments

@YakDriver
Copy link
Member

Terraform Core Version

1.10.5

AWS Provider Version

5.83.0

Affected Resource(s)

  • aws_cloudformation_stack

Expected Behavior

Stack should create without error

Actual Behavior

Stack create fails with an error

Relevant Error/Panic Output Snippet

Error: yaml: unmarshal errors: line 438: mapping key "S3OriginConfig" already defined at line 435  
with aws_cloudformation_stack.cloudfront-s3-stack  
on cloudfront-setup.tf line 36, in resource "aws_cloudformation_stack" "cloudfront-s3-stack":  
resource "aws_cloudformation_stack" "cloudfront-s3-stack" {

Terraform Configuration Files

resource "aws_cloudformation_stack" "website" {
  name          = "cloudfront-s3-stack"
  template_body = file("171977cloudfront-s3duplicated.yaml")

  parameters = {
    DefaultRootObject                 = "index.html"
    DnsName                           = "example.com"
    SsmBucketName                     = "my-ssm-bucket-name"
    S3BucketName                      = "my-s3-bucket-name"
    S3BucketLogName                   = "my-s3-log-bucket"
    AcmCertificateArn                 = "arn:aws:acm:us-east-1:123456789012:certificate/example"
    ErrorPagePath                     = "/404.html"
    Application                       = "MyApp"
    Env                               = "dev"
    AuthLambdaArn                     = "arn:aws:lambda:us-east-1:123456789012:function:my-auth-lambda"
    S3BucketCnpReceipt                = "my-cnp-receipt-bucket"
    S3BucketCnpReceiptDomainName      = "my-cnp-receipt-domain"
    WebaclArn                         = "arn:aws:wafv2:us-east-1:123456789012:global/webacl/example"
    Stage                             = "dev"
    SnsArn                            = "arn:aws:sns:us-east-1:123456789012:example-topic"
    KinesisStreamArn                  = "arn:aws:kinesis:us-east-1:123456789012:stream/example"
    RealTimeLogRoleArn                = "arn:aws:iam::123456789012:role/example-role"
  }

  tags = {
    Application = "MyApp"
    Environment = "dev"
    Team        = "DevOps"
  }
}
AWSTemplateFormatVersion: 2010-09-09

Description: Static website hosting with S3 and CloudFront

Parameters:
  DefaultRootObject:
    Description: The default path for the index document.
    Type: String
    Default: index.html

  DnsName:
    Type: String
    Description: The website domain name.

  SsmBucketName:
    Type: String
    Description: SSM keyname for S3 bucket for  website hosting

  S3BucketName:
    Type: String
    Description: Bucketname for website hosting S3

  S3BucketLogName:
    Type: String
    Description: S3 Bucketname for storing cloudfront  logs

  AcmCertificateArn:
    Type: AWS::SSM::Parameter::Value<String>
    Description: The ARN of the SSL certificate to use for the CloudFront distribution

  ErrorPagePath:
    Description: The path of the error page for the website (e.g. /error.html)
    Type: String
    Default: /404.html

  Application:
    Description: Name of the Application
    Type: String

  Env:
    Description: Type of the Environment
    Type: String

  AuthLambdaArn:
    Description: Auth lambda Arn for cloudfront authentication
    Type: String
    #Type: AWS::SSM::Parameter::Value<String>
  S3BucketCnpReceipt:
    Type: String
  S3BucketCnpReceiptDomainName:
    Type: String

  WebaclArn:
    Description: global web acl arn
    Type: String

  Stage:
    Description: Name of the Stage
    Type: String

  SnsArn:
    Type: AWS::SSM::Parameter::Value<String>
    Description: The ARN of the SNS Topic to get the notification from S3 Bucket

  KinesisStreamArn:
    Type: String
    Description: The Kinesis stream arn to collect the real time logs.

  RealTimeLogRoleArn:
    Type: String
    Description: The iam role arn to push the logs to kinesis stream.

Conditions:
  NoAuthentication: !Equals [!Ref Env, prod]
  CreateAuthentication: !Not [!Equals [!Ref Env, prod]]

Resources:

  S3BucketWebsite:
    Type: "AWS::S3::Bucket"
    Properties:
      BucketName: !Ref S3BucketName
      LoggingConfiguration:
        DestinationBucketName: !Ref S3BucketLoggingCloudFront
        LogFilePrefix: !Join
          - ""
          - - !Ref "AWS::StackName"
            - /s3/
      VersioningConfiguration:
        Status: Enabled
      LifecycleConfiguration:
        Rules:
          - Id: NonCurrentVersionRule
            Status: Enabled 
            NoncurrentVersionExpirationInDays: 1095
      BucketEncryption:
        ServerSideEncryptionConfiguration:
        - ServerSideEncryptionByDefault:
            SSEAlgorithm: AES256
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true
      NotificationConfiguration:
        TopicConfigurations:
          - Topic: !Ref SnsArn
            Event: 's3:ObjectRemoved:*'

  ReadPolicy:
    Type: "AWS::S3::BucketPolicy"
    Properties:
      Bucket: !Ref S3BucketWebsite
      PolicyDocument:
        Statement:
          - Action: "s3:GetObject"
            Effect: Allow
            Resource: !Sub "arn:aws:s3:::${S3BucketWebsite}/*"
            Principal:
              CanonicalUser: !GetAtt CloudFrontOriginAccessIdentity.S3CanonicalUserId
          - Action: 's3:*'
            Effect: Deny
            Principal: '*'
            Resource:
              - !Join
                - ''
                - - 'arn:'
                  - !Ref 'AWS::Partition'
                  - ':s3:::'
                  - !Ref S3BucketWebsite
                  - /*
            Condition:
              Bool:
                'aws:SecureTransport': false

  S3BucketLoggingCloudFront:
      Type: "AWS::S3::Bucket"
      Properties:
        BucketName: !Ref S3BucketLogName
        AccessControl: LogDeliveryWrite
        Tags:
          - Key: Application
            Value: log storage
          - Key: STAGE
            Value: !Ref Stage
        VersioningConfiguration:
          Status: Enabled
        LifecycleConfiguration:
          Rules:
            - Id: NonCurrentVersionRule
              Status: Enabled 
              NoncurrentVersionExpirationInDays: 1095
        BucketEncryption:
          ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: AES256
        PublicAccessBlockConfiguration:
          BlockPublicAcls: true
          BlockPublicPolicy: true
          IgnorePublicAcls: true
          RestrictPublicBuckets: true
        NotificationConfiguration:
          TopicConfigurations:
            - Topic: !Ref SnsArn
              Event: 's3:ObjectRemoved:*'

  ReadWritePolicy:
      Type: "AWS::S3::BucketPolicy"
      Properties:
        Bucket: !Ref S3BucketLoggingCloudFront
        PolicyDocument:
          Statement:
            - Action:
                - "s3:GetObject"
                - "s3:PutObject"
              Effect: Allow
              Resource: !Sub "arn:aws:s3:::${S3BucketLoggingCloudFront}/*"
              Principal:
                CanonicalUser: !GetAtt CloudFrontOriginAccessIdentity.S3CanonicalUserId
            - Action: 's3:*'
              Effect: Deny
              Principal: '*'
              Resource:
                - !Join
                  - ''
                  - - 'arn:'
                    - !Ref 'AWS::Partition'
                    - ':s3:::'
                    - !Ref S3BucketLoggingCloudFront
                    - /*
              Condition:
                Bool:
                  'aws:SecureTransport': false

  SsmParameterS3BucketName:
    Type: "AWS::SSM::Parameter"
    Properties:
      Name: !Ref SsmBucketName
      Type: String
      Value: !Ref S3BucketWebsite

  CloudFrontOriginAccessIdentity:
    Type: "AWS::CloudFront::CloudFrontOriginAccessIdentity"
    Properties:
      CloudFrontOriginAccessIdentityConfig:
        Comment: !Ref S3BucketWebsite

  CloudFrontOriginAccessIdentityCnpReceipt:
    Type: "AWS::CloudFront::CloudFrontOriginAccessIdentity"
    Properties:
      CloudFrontOriginAccessIdentityConfig:
        Comment: !Ref S3BucketCnpReceipt

  CloudFrontDistribution:
    Type: "AWS::CloudFront::Distribution"
    Condition: NoAuthentication
    Properties:
      Tags:
        - Key: Application
          Value: !Ref Application
        - Key: Environment
          Value: !Ref Env
        - Key: STAGE
          Value: !Ref Stage
      DistributionConfig:
        Logging:
          IncludeCookies: "false"
          Bucket: !GetAtt S3BucketLoggingCloudFront.DomainName
          Prefix: CfLog
        Aliases: !Split [",", !Ref DnsName]
        CustomErrorResponses:
          - ErrorCode: 403
            ResponseCode: 404
            ResponsePagePath: !Ref ErrorPagePath
        CacheBehaviors:
          - TargetOriginId: !Join
                    - '-'
                    -  - 'S3'
                       -  !Ref S3BucketCnpReceipt
            PathPattern: 'pdf/*'
            CachePolicyId: "4135ea2d-6df8-44a3-9df3-4b5a84be39ad"
            ViewerProtocolPolicy: redirect-to-https
            ResponseHeadersPolicyId: !Ref ResponseHeadersPolicy
            RealtimeLogConfigArn: !GetAtt CloudfrontRTLogsConfig.Arn
            MinTTL: 0
            AllowedMethods:
              - HEAD
              - GET
              - OPTIONS
            CachedMethods:
              - HEAD
              - GET
              - OPTIONS
            Compress: true
            ForwardedValues:
                Cookies:
                    Forward: none
                QueryString: false
          - TargetOriginId: !Join
                    - '-'
                    -  - 'S3'
                       -  !Ref S3BucketCnpReceipt
            PathPattern: 'png/*'
            CachePolicyId: "4135ea2d-6df8-44a3-9df3-4b5a84be39ad"
            ViewerProtocolPolicy: redirect-to-https
            ResponseHeadersPolicyId: !Ref ResponseHeadersPolicy
            RealtimeLogConfigArn: !GetAtt CloudfrontRTLogsConfig.Arn
            MinTTL: 0
            AllowedMethods:
              - HEAD
              - GET
              - OPTIONS
            CachedMethods:
              - HEAD
              - GET
              - OPTIONS
            Compress: true
            ForwardedValues:
                Cookies:
                    Forward: none
                QueryString: false
        DefaultCacheBehavior:
            TargetOriginId: s3origin
            AllowedMethods:
                - GET
                - HEAD
                - OPTIONS
            CachedMethods:
                - GET
                - HEAD
                - OPTIONS
            Compress: true
            DefaultTTL: 3600
            ForwardedValues:
                Cookies:
                    Forward: none
                QueryString: false
            MaxTTL: 86400
            MinTTL: 60
            ViewerProtocolPolicy: redirect-to-https
            ResponseHeadersPolicyId: !Ref ResponseHeadersPolicy
            RealtimeLogConfigArn: !GetAtt CloudfrontRTLogsConfig.Arn
        DefaultRootObject: !Ref DefaultRootObject
        Enabled: true
        HttpVersion: http2
        Origins:
          - DomainName: !GetAtt S3BucketWebsite.DomainName
            Id: s3origin
            S3OriginConfig:
              OriginAccessIdentity: !Sub >-
                origin-access-identity/cloudfront/${CloudFrontOriginAccessIdentity}
        PriceClass: PriceClass_All
        ViewerCertificate:
          AcmCertificateArn: !Ref AcmCertificateArn
          MinimumProtocolVersion: TLSv1.2_2021
          SslSupportMethod: sni-only
        WebACLId: !Ref WebaclArn


  CloudFrontDistributionAuth:
    Type: "AWS::CloudFront::Distribution"
    Condition: CreateAuthentication
    Properties:
      Tags:
        - Key: Application
          Value: !Ref Application
        - Key: Environment
          Value: !Ref Env
        - Key: STAGE
          Value: !Ref Stage
      DistributionConfig:
        Logging:
          IncludeCookies: "false"
          Bucket: !GetAtt S3BucketLoggingCloudFront.DomainName
          Prefix: CfLog
        Aliases: !Split [",", !Ref DnsName]
        CustomErrorResponses:
          - ErrorCode: 403
            ResponseCode: 404
            ResponsePagePath: !Ref ErrorPagePath
        CacheBehaviors:
          - TargetOriginId: !Join
                    - '-'
                    -  - 'S3'
                       -  !Ref S3BucketCnpReceipt
            PathPattern: 'pdf/*'
            CachePolicyId: "4135ea2d-6df8-44a3-9df3-4b5a84be39ad"
            ViewerProtocolPolicy: redirect-to-https
            ResponseHeadersPolicyId: !Ref ResponseHeadersPolicy
            RealtimeLogConfigArn: !GetAtt CloudfrontRTLogsConfig.Arn
            MinTTL: 0
            AllowedMethods:
              - HEAD
              - GET
              - OPTIONS
            CachedMethods:
              - HEAD
              - GET
              - OPTIONS
            Compress: true
            ForwardedValues:
                Cookies:
                    Forward: none
                QueryString: false
          - TargetOriginId: !Join
                    - '-'
                    -  - 'S3'
                       -  !Ref S3BucketCnpReceipt
            PathPattern: 'png/*'
            CachePolicyId: "4135ea2d-6df8-44a3-9df3-4b5a84be39ad"
            ViewerProtocolPolicy: redirect-to-https
            ResponseHeadersPolicyId: !Ref ResponseHeadersPolicy
            RealtimeLogConfigArn: !GetAtt CloudfrontRTLogsConfig.Arn
            MinTTL: 0
            AllowedMethods:
              - HEAD
              - GET
              - OPTIONS
            CachedMethods:
              - HEAD
              - GET
              - OPTIONS
            Compress: true
            ForwardedValues:
                Cookies:
                    Forward: none
                QueryString: false
        DefaultCacheBehavior:
            TargetOriginId: s3origin
            AllowedMethods:
                - GET
                - HEAD
                - OPTIONS
            CachedMethods:
                - GET
                - HEAD
                - OPTIONS
            Compress: true
            DefaultTTL: 3600
            ForwardedValues:
                Cookies:
                    Forward: none
                QueryString: false
            MaxTTL: 86400
            MinTTL: 60
            ViewerProtocolPolicy: redirect-to-https
            ResponseHeadersPolicyId: !Ref ResponseHeadersPolicy
            RealtimeLogConfigArn: !GetAtt CloudfrontRTLogsConfig.Arn
            LambdaFunctionAssociations:
              - EventType: viewer-request
                IncludeBody: false
                LambdaFunctionARN: !Ref AuthLambdaArn
        DefaultRootObject: !Ref DefaultRootObject
        Enabled: true
        HttpVersion: http2
        Origins:
          - DomainName: !GetAtt S3BucketWebsite.DomainName
            Id: s3origin
            S3OriginConfig:
              OriginAccessIdentity: !Sub >-
                origin-access-identity/cloudfront/${CloudFrontOriginAccessIdentity}
          - DomainName: !Ref S3BucketCnpReceiptDomainName
            Id:  !Join
                - '-'
                -  - 'S3'
                   -  !Ref S3BucketCnpReceipt
            S3OriginConfig:
              OriginAccessIdentity: !Sub >-
                origin-access-identity/cloudfront/${CloudFrontOriginAccessIdentityCnpReceipt}
        PriceClass: PriceClass_All
        ViewerCertificate:
          AcmCertificateArn: !Ref AcmCertificateArn
          MinimumProtocolVersion: TLSv1.2_2021
          SslSupportMethod: sni-only
        WebACLId: !Ref WebaclArn

  ResponseHeadersPolicy:
    Type: AWS::CloudFront::ResponseHeadersPolicy
    Properties: 
      ResponseHeadersPolicyConfig: 
        Name: !Join ["-", [!Ref Env, '-', !Ref Application, '-security-headers']]
        CorsConfig:
          AccessControlAllowCredentials: false
          AccessControlAllowHeaders:
            Items:
              - "*"
          AccessControlAllowOrigins:
            Items:
              - "*"
          AccessControlAllowMethods:
            Items:
              - ALL
          AccessControlExposeHeaders:
            Items:
              - "-"
          AccessControlMaxAgeSec: 600
          OriginOverride: false
        SecurityHeadersConfig: 
          StrictTransportSecurity: 
            AccessControlMaxAgeSec: 31536000
            Override: false
          ContentTypeOptions: 
            Override: true
          FrameOptions:
            FrameOption: SAMEORIGIN
            Override: false
          ReferrerPolicy: 
            ReferrerPolicy: "strict-origin-when-cross-origin"
            Override: false
          XSSProtection: 
            ModeBlock: true
            Protection: true
            Override: false 
          ContentSecurityPolicy:
            
            Override: false

  CloudfrontRTLogsConfig:
    Type: AWS::CloudFront::RealtimeLogConfig
    Properties: 
      EndPoints:
        - KinesisStreamConfig:
            RoleArn: !Ref RealTimeLogRoleArn 
            StreamArn: !Ref KinesisStreamArn
          StreamType: Kinesis
      Fields: 
        - timestamp
        - c-ip
        - cs-host
        - cs-uri-stem
        - cs-headers
        - cs-method
        - cs-protocol
        - cs-bytes
        - cs-protocol-version
        - c-ip-version
        - ssl-protocol
        - c-port
        - c-country
      Name: cloudfront-cnp-logs
      SamplingRate: 100

  SsmParameterDomainAuth:
    Type: "AWS::SSM::Parameter"
    Condition: CreateAuthentication
    Properties:
      Name: !Sub "${AWS::StackName}-DomainName"
      Type: String
      Value: !GetAtt CloudFrontDistributionAuth.DomainName

  SsmParameterDomain:
    Type: "AWS::SSM::Parameter"
    Condition: NoAuthentication
    Properties:
      Name: !Sub "${AWS::StackName}-DomainName"
      Type: String
      Value: !GetAtt CloudFrontDistribution.DomainName

  SsmParameterDistributionId:
    Type: "AWS::SSM::Parameter"
    Condition: NoAuthentication
    Properties:
      Name: !Join ["", [ "/", !Ref Env, "/", !Ref Application, "/CNP_CLOUDFRONT_DISTRIBUTION_ID"]] 
      Type: String
      Value: !Ref CloudFrontDistribution

  SsmParameterDistributionIdAuth:
    Type: "AWS::SSM::Parameter"
    Condition: CreateAuthentication
    Properties:
      Name: !Join ["", [ "/", !Ref Env, "/", !Ref Application, "/CNP_CLOUDFRONT_DISTRIBUTION_ID"]]
      Type: String
      Value: !Ref CloudFrontDistributionAuth

  SsmParameterS3User:
    Condition: NoAuthentication
    Type: AWS::SSM::Parameter
    Properties:
      Name: !Sub "${AWS::StackName}-receipt-s3-canonical-user-id"
      Type: String
      Value: !GetAtt CloudFrontOriginAccessIdentityCnpReceipt.Id
      Description: s3 bucket name for receipt

  SsmParameterS3UserAuth:
    Condition: CreateAuthentication
    Type: AWS::SSM::Parameter
    Properties:
      Name: !Sub "${AWS::StackName}-receipt-s3-canonical-user-id"
      Type: String
      Value: !GetAtt CloudFrontOriginAccessIdentityCnpReceipt.Id
      Description: s3 bucket name for receipt

Outputs:
  BucketName:
    Description: S3 Bucket Name
    Value: !Ref S3BucketWebsite
    Export:
      Name: !Sub "${AWS::StackName}-BucketName"

  DistributionId:
    Description: CloudFront Distribution ID
    Value: !Ref CloudFrontDistribution
    Condition: NoAuthentication
    Export:
      Name: !Sub "${AWS::StackName}-DistributionId"

  Domain:
    Description: Cloudfront Domain
    Value: !GetAtt CloudFrontDistribution.DomainName
    Condition: NoAuthentication
    Export:
      Name: !Sub "${AWS::StackName}-DomainName"

  ConUser:
    Description: Cloudfront Domain
    Value: !GetAtt CloudFrontOriginAccessIdentityCnpReceipt.Id
    Condition: NoAuthentication
    Export:
      Name: !Sub "${AWS::StackName}-receipt-s3-canonical-user-id"

  DistributionIdAuth:
    Description: CloudFront Distribution ID
    Value: !Ref CloudFrontDistributionAuth
    Condition: CreateAuthentication
    Export:
      Name: !Sub "${AWS::StackName}-DistributionId"

  DomainAuth:
    Description: Cloudfront Domain
    Value: !GetAtt CloudFrontDistributionAuth.DomainName
    Condition: CreateAuthentication
    Export:
      Name: !Sub "${AWS::StackName}-DomainName"

  ConUserAuth:
    Description: Cloudfront Domain
    Value: !GetAtt CloudFrontOriginAccessIdentityCnpReceipt.Id
    Condition: CreateAuthentication
    Export:
      Name: !Sub "${AWS::StackName}-receipt-s3-canonical-user-id"

Steps to Reproduce

  1. terraform apply

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

Would you like to implement a fix?

None
@YakDriver YakDriver added the bug Addresses a defect in current functionality. label Feb 5, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 5, 2025

Community Note

Voting for Prioritization

  • Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

  • If you are interested in working on this issue, please leave a comment.
  • If this would be your first contribution, please review the contribution guide.

@github-actions github-actions bot added the service/cloudformation Issues and PRs that pertain to the cloudformation service. label Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Addresses a defect in current functionality. service/cloudformation Issues and PRs that pertain to the cloudformation service.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@YakDriver