aws-codepipeline-actions: Problem Cloning the AWS CDK Repository on Windows #32808

azarboon · 2025-01-09T05:59:31Z

Describe the bug

When cloning the AWS CDK repository on a Windows system, I encountered issues where certain files appear in the Git index but are not present on the filesystem. This causes git status to show these files as deleted, and attempts to restore or interact with them fail. The issue might be related to Windows' filesystem limitations, but the root cause remains unclear.

To fix the issue, I cloned the repository using Windows Subsystem for Linux (WSL) in VS Code, the Bash terminal in VS Code, and Git Desktop's Bash, but none of these helped. I also changed case sensitivity (through core.ignorecase), enabled long file paths (through core.longpaths), and modified the Windows registry, but this did not resolve the problem.

Regression Issue

Select this option if this issue appears to be a regression.

Last Known Working CDK Version

No response

Expected Behavior

The repository should clone successfully on Windows, with all files checked out to the filesystem as expected. Running git status and git diff after merging updates from upstream/main should not show any unexpected deletions or errors related to file handling. Cross-platform compatibility issues, if present, should be handled gracefully to allow contributors to work seamlessly regardless of their operating system.

Current Behavior

git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.assets.json
        deleted:    packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.template.json

no changes added to commit (use "git add" and/or "git commit -a")

Also,

git diff
diff --git a/packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.assets.json b/packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.assets.json
deleted file mode 100644
index 8352392771..0000000000
--- a/packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.assets.json
+++ /dev/null
@@ -1,20 +0,0 @@
-{
-  "version": "38.0.1",
-  "files": {
-    "6ead11ac2a258f5956aa4d821020d99d8460173de308c7fd28a388dc243e3f89": {
-      "source": {
-        "path": "cross-region-stack-123456789012:us-west-2.template.json",
-        "packaging": "file"
-      },
-      "destinations": {
-        "123456789012-us-west-2": {
-          "bucketName": "cdk-hnb659fds-assets-123456789012-us-west-2",
-          "objectKey": "6ead11ac2a258f5956aa4d821020d99d8460173de308c7fd28a388dc243e3f89.json",
-          "region": "us-west-2",
-          "assumeRoleArn": "arn:${AWS::Partition}:iam::123456789012:role/cdk-hnb659fds-file-publishing-role-123456789012-us-west-2"
-        }
-      }
-    }
-  },
-  "dockerImages": {}
-}
\ No newline at end of file
diff --git a/packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.template.json b/packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.template.json
deleted file mode 100644
index d37e16bccf..0000000000
--- a/packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.template.json
+++ /dev/null
@@ -1,167 +0,0 @@
-{
- "Resources": {
-  "CrossRegionCodePipelineReplicationBucketEncryptionKey70216490": {
-   "Type": "AWS::KMS::Key",
-   "Properties": {
-    "KeyPolicy": {
-     "Statement": [
-      {
-       "Action": "kms:*",
-       "Effect": "Allow",
-       "Principal": {
-        "AWS": "arn:aws:iam::123456789012:root"
-       },
-       "Resource": "*"
-      },
-      {
-       "Action": [
-        "kms:Decrypt",
-        "kms:DescribeKey",
-        "kms:Encrypt",
-        "kms:GenerateDataKey*",
-        "kms:ReEncrypt*"
-       ],
-       "Effect": "Allow",
-       "Principal": {
-        "AWS": "arn:aws:iam::123456789012:role/MyPipelineRoleName"
-       },
-       "Resource": "*"
-      }
-     ],
-     "Version": "2012-10-17"
-    }
-   },
-   "UpdateReplacePolicy": "Delete",
-   "DeletionPolicy": "Delete"
-  },
-  "CrossRegionCodePipelineReplicationBucketEncryptionAliasF1A0F37D": {
-   "Type": "AWS::KMS::Alias",
-   "Properties": {
-    "AliasName": "alias/ort-us-wtencryptionalias8f9701ce6a32f909886f",
-    "TargetKeyId": {
-     "Fn::GetAtt": [
-      "CrossRegionCodePipelineReplicationBucketEncryptionKey70216490",
-      "Arn"
-     ]
-    }
-   },
-   "UpdateReplacePolicy": "Delete",
-   "DeletionPolicy": "Delete"
-  },
-  "CrossRegionCodePipelineReplicationBucketFC3227F2": {
-   "Type": "AWS::S3::Bucket",
-   "Properties": {
-    "BucketEncryption": {
-     "ServerSideEncryptionConfiguration": [
-      {
-       "ServerSideEncryptionByDefault": {
-        "KMSMasterKeyID": {
-         "Fn::Join": [
-          "",
-          [
-           "arn:aws:kms:us-west-2:123456789012:",
-           {
-            "Ref": "CrossRegionCodePipelineReplicationBucketEncryptionAliasF1A0F37D"  
-           }
-          ]
-         ]
-        },
-        "SSEAlgorithm": "aws:kms"
-       }
-      }
-     ]
-    },
-    "BucketName": "cross-region-support-us-weplicationbucket8a287d3945436008ebfd",    
-    "PublicAccessBlockConfiguration": {
-     "BlockPublicAcls": true,
-     "BlockPublicPolicy": true,
-     "IgnorePublicAcls": true,
-     "RestrictPublicBuckets": true
-    }
-   },
-   "UpdateReplacePolicy": "Retain",
-   "DeletionPolicy": "Retain"
-  },
-  "CrossRegionCodePipelineReplicationBucketPolicyB7BA2BCA": {
-   "Type": "AWS::S3::BucketPolicy",
-   "Properties": {
-    "Bucket": {
-     "Ref": "CrossRegionCodePipelineReplicationBucketFC3227F2"
-    },
-    "PolicyDocument": {
-     "Statement": [
-      {
-       "Action": "s3:*",
-       "Condition": {
-        "Bool": {
-         "aws:SecureTransport": "false"
-        }
-       },
-       "Effect": "Deny",
-       "Principal": {
-        "AWS": "*"
-       },
-       "Resource": [
-        {
-         "Fn::GetAtt": [
-          "CrossRegionCodePipelineReplicationBucketFC3227F2",
-          "Arn"
-         ]
-        },
-        {
-         "Fn::Join": [
-          "",
-          [
-           {
-            "Fn::GetAtt": [
-             "CrossRegionCodePipelineReplicationBucketFC3227F2",
-             "Arn"
-            ]
-           },
-           "/*"
-          ]
-         ]
-        }
-       ]
-      }
-     ],
-     "Version": "2012-10-17"
-    }
-   }
-  }
- },
- "Parameters": {
-  "BootstrapVersion": {
-   "Type": "AWS::SSM::Parameter::Value<String>",
-   "Default": "/cdk-bootstrap/hnb659fds/version",
-   "Description": "Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"
-  }
- },
- "Rules": {
-  "CheckBootstrapVersion": {
-   "Assertions": [
-    {
-     "Assert": {
-      "Fn::Not": [
-       {
-        "Fn::Contains": [
-         [
-          "1",
-          "2",
-          "3",
-          "4",
-          "5"
-         ],
-         {
-          "Ref": "BootstrapVersion"
-         }
-        ]
-       }
-      ]
-     },
-     "AssertDescription": "CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."
-    }
-   ]
-  }
- }
-}
\ No newline at end of file

Also,

git restore "packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.assets.json"
git restore "packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.template.json"

error: unable to stat just-written file packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.assets.json: No such file or directory
error: unable to stat just-written file packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.template.json: No such file or directory

Reproduction Steps

# Fork the AWS CDK repository on GitHub
# Clone the forked repository
git clone https://github.com/<your-username>/aws-cdk.git
cd aws-cdk

# Add the upstream repository to fetch updates
git remote add upstream https://github.com/aws/aws-cdk.git

# Fetch the latest changes from upstream and merge them
git fetch upstream
git merge upstream/main

# Check the status to see any changes
git status

# View differences in the repository
git diff

# Attempt to restore problematic files
git restore <file>

Possible Solution

Investigate and address why the files cannot be created on Windows systems.
Consider renaming files with incompatible characters (e.g., replacing colons with _ or -).
Provide a script or process to handle these files for Windows users specifically.

Additional Information/Context

No response

CDK CLI Version

2.173.2 (build f8e6207)

Framework Version

No response

Node.js Version

v22.9.0

OS

Windows 11 Home Single Language - version 23H2 - OS build 22631.4602

Language

TypeScript
but seems this issue is agnostic of language.

Language Version

No response

Other information

Git: Version 2.42
Cloning method: Git Bash / WSL / GitHub Desktop
Repository: AWS CDK (main branch)

The text was updated successfully, but these errors were encountered:

azarboon · 2025-01-09T07:19:42Z

For anyone who encounter this issue, here is a temporarily:

git clone https://github.com/your-username/aws-cdk3.git 
cd aws-cdk3 
git remote add upstream https://github.com/aws/aws-cdk.git 
git fetch upstream 
git checkout main 
git merge upstream/main 
echo "packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/" >> .git/info/exclude 
git rm --cached packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.assets.json && git rm --cached packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.template.json 
git commit -m "Remove problematic files from git index" 
git push origin main 
git checkout -b your-branch-name

ashishdhingra · 2025-01-13T21:50:46Z

Got below error while cloning on Windows:

Cloning into 'D:\source\GitHub\aws-cdk'...
remote: Enumerating objects: 396735, done.        
remote: Counting objects: 100% (111/111), done.        
remote: Compressing objects: 100% (72/72), done.        
remote: Total 396735 (delta 59), reused 41 (delta 39), pack-reused 396624 (from 2)        
Receiving objects: 100% (396735/396735), 1.06 GiB | 18.14 MiB/s, done.
Resolving deltas: 100% (322693/322693), done.
error: invalid path 'packages/@aws-cdk-testing/framework-integ/test/aws-codepipeline-actions/test/integ.codepipeline-with-nested-stack.js.snapshot/cross-region-stack-123456789012:us-west-2.assets.json'
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'


Would you like to retry cloning ?

Character : is not a valid path for Windows.
Need to reach out to CDK team for inputs. CC @khushail

khushail · 2025-01-14T01:27:38Z

Seeking Team's insights on this issue for further analysis.

samson-keung · 2025-01-15T21:45:36Z

I looked into why the repo has files with : in the file name. Here is my findings:

The codepipeline.Pipeline construct contains code that creates a nested stack. When it creates the nested stack, it assign a stack id to the stack, the id has the : character:

aws-cdk/packages/aws-cdk-lib/aws-codepipeline/lib/pipeline.ts

Line 826 in a09ccdb

    
           const supportStackId = `cross-region-stack-${this.reuseCrossRegionSupportStacks ? pipelineAccount : pipelineStack.stackName}:${actionRegion}`;

Then when the CDK app get synthesized, the generate template file name is based on the stack id, hence, inherited the : char.

Looking at the code, I do not see a prop on codepipeline.Pipeline to override the id for the nested stack. And knowing that : in file name is not supported in Windows, I think it was a mistake to use : in the stack id. I think the solution is to introduce a feature flag which brach the code to use a different id format.

azarboon · 2025-01-16T11:52:48Z

I looked into why the repo has files with : in the file name. Here is my findings:

The codepipeline.Pipeline construct contains code that creates a nested stack. When it creates the nested stack, it assign a stack id to the stack, the id has the : character:

aws-cdk/packages/aws-cdk-lib/aws-codepipeline/lib/pipeline.ts

Line 826 in a09ccdb

const supportStackId = cross-region-stack-${this.reuseCrossRegionSupportStacks ? pipelineAccount : pipelineStack.stackName}:${actionRegion};
Then when the CDK app get synthesized, the generate template file name is based on the stack id, hence, inherited the : char.

Looking at the code, I do not see a prop on codepipeline.Pipeline to override the id for the nested stack. And knowing that : in file name is not supported in Windows, I think it was a mistake to use : in the stack id. I think the solution is to introduce a feature flag which brach the code to use a different id format.

@samson-keung Thanks for this. I'm wondering why are you suggesting feature flag. May I make a PR and just replace : with -?

samson-keung · 2025-01-16T17:11:14Z

@azarboon It has to be behind feature flag because changing the id of the nested stack can cause behavioural changes to existing CDK apps. E.g. the nested stack being replaced or a new stack being created. There may be other cases where CDK users are relying on reading the stack id.

A reference in the Contribution Guide about breaking changes: https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md#breaking-changes

Please feel free to open a PR! Contributions are always welcome.

azarboon · 2025-01-17T02:30:19Z

@azarboon It has to be behind feature flag because changing the id of the nested stack can cause behavioural changes to existing CDK apps. E.g. the nested stack being replaced or a new stack being created. There may be other cases where CDK users are relying on reading the stack id.

A reference in the Contribution Guide about breaking changes: https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md#breaking-changes

Please feel free to open a PR! Contributions are always welcome.

@samson-keung Thank you for your response. However, I am currently facing some challenges with creating pull requests. I have detailed the issue in #32910. Could you please assist me with resolving this? Once the issue is addressed, I will proceed with the task.

github-actions · 2025-03-21T18:49:59Z

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

github-actions · 2025-03-21T18:50:04Z

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

…IDs (aws#33863) ### Issue Closes aws#32808 Minimal reproduction: ```ts const app = new cdk.App(); const stack = new cdk.Stack(app, 'Stack:Test', { stackName: "valid" }); assert(!stack.artifactId.match(/[^A-Za-z0-9_\-\.]/g)) ``` ### Reason for this change When users create a `Stack` with an id containing certain special characters, the produced artifact file name would contain invalid characters. ### Description of changes Ensure only valid artifact files names are produces, no matter the input. ### Describe any new or updated permissions being added n/a ### Description of how you validated changes New unit tests, existing test suite I have updated the integ test using `--dry-run` as per instructions in the test file. I have deployed the integ test using the instructions in the test file. ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

azarboon added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jan 9, 2025

github-actions bot added the @aws-cdk/assets Related to the @aws-cdk/assets package label Jan 9, 2025

azarboon changed the title ~~aws-cdk-testing: Problem Cloning the AWS CDK Repository on Windows~~ aws-codepipeline-actions: Problem Cloning the AWS CDK Repository on Windows Jan 9, 2025

github-actions bot added the @aws-cdk/aws-codepipeline-actions label Jan 9, 2025

azarboon mentioned this issue Jan 9, 2025

chore: add empty file to .gitignore to prevent cloning issues on Windows #32810

Closed

1 task

khushail added needs-reproduction This issue needs reproduction. p2 and removed needs-triage This issue or PR still needs to be triaged. labels Jan 9, 2025

khushail self-assigned this Jan 9, 2025

khushail added p1 effort/small Small work item – less than a day of effort and removed p2 needs-reproduction This issue needs reproduction. labels Jan 13, 2025

ashishdhingra mentioned this issue Jan 14, 2025

Contributing Guidelines: Ongoing Issue with 'Out-of-Date with the Base Branch' #32910

Closed

khushail removed their assignment Jan 14, 2025

azarboon mentioned this issue Jan 19, 2025

fix(core): remove colon to ensure compatibility with Windows users (under feature flag) #33014

Closed

1 task

github-actions bot mentioned this issue Feb 1, 2025

Monthly issue metrics report #33265

Closed

mrgrain added @aws-cdk/aws-codepipeline Related to AWS CodePipeline and removed @aws-cdk/assets Related to the @aws-cdk/assets package @aws-cdk/aws-codepipeline-actions labels Feb 28, 2025

paulhcsun self-assigned this Mar 13, 2025

shikha372 unassigned paulhcsun Mar 17, 2025

shikha372 assigned scorbiere Mar 17, 2025

paulhcsun unassigned scorbiere Mar 18, 2025

mrgrain mentioned this issue Mar 21, 2025

fix(core): does not generate a valid artifact id from some construct IDs #33863

Merged

1 task

mrgrain added @aws-cdk/core Related to core CDK functionality and removed @aws-cdk/aws-codepipeline Related to AWS CodePipeline labels Mar 21, 2025

mergify bot closed this as completed in #33863 Mar 21, 2025

mergify bot closed this as completed in 6a5638a Mar 21, 2025

github-actions bot locked as resolved and limited conversation to collaborators Mar 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

aws-codepipeline-actions: Problem Cloning the AWS CDK Repository on Windows #32808

aws-codepipeline-actions: Problem Cloning the AWS CDK Repository on Windows #32808

azarboon commented Jan 9, 2025 •

edited

Loading

azarboon commented Jan 9, 2025

ashishdhingra commented Jan 13, 2025 •

edited

Loading

khushail commented Jan 14, 2025

samson-keung commented Jan 15, 2025

azarboon commented Jan 16, 2025 •

edited

Loading

samson-keung commented Jan 16, 2025 •

edited

Loading

azarboon commented Jan 17, 2025

github-actions bot commented Mar 21, 2025

github-actions bot commented Mar 21, 2025

aws-codepipeline-actions: Problem Cloning the AWS CDK Repository on Windows #32808

aws-codepipeline-actions: Problem Cloning the AWS CDK Repository on Windows #32808

Comments

azarboon commented Jan 9, 2025 • edited Loading

Describe the bug

Regression Issue

Last Known Working CDK Version

Expected Behavior

Current Behavior

Reproduction Steps

Possible Solution

Additional Information/Context

CDK CLI Version

Framework Version

Node.js Version

OS

Language

Language Version

Other information

azarboon commented Jan 9, 2025

ashishdhingra commented Jan 13, 2025 • edited Loading

khushail commented Jan 14, 2025

samson-keung commented Jan 15, 2025

azarboon commented Jan 16, 2025 • edited Loading

samson-keung commented Jan 16, 2025 • edited Loading

azarboon commented Jan 17, 2025

github-actions bot commented Mar 21, 2025

github-actions bot commented Mar 21, 2025

azarboon commented Jan 9, 2025 •

edited

Loading

ashishdhingra commented Jan 13, 2025 •

edited

Loading

azarboon commented Jan 16, 2025 •

edited

Loading

samson-keung commented Jan 16, 2025 •

edited

Loading