Unable to use aws s3 sync

[h0806449f]

Describe the bug

aws s3 sync s3://test-bucket-for-sync-source s3://test-bucket-for-sync-destination

can not proceed without turn on the "Transfer Acceleration"

Expected Behavior

copy s3://test-bucket-for-sync-source to s3://test-bucket-for-sync-destination

and all object will from source bucket to destination bucket

Current Behavior

fatal error: An error occurred (InvalidRequest) when calling the ListObjectsV2 operation: S3 Transfer Acceleration is not configured on this bucket

Reproduction Steps

aws s3 sync s3://test-bucket-for-sync-source s3://test-bucket-for-sync-destination

Possible Solution

turn on Transfer acceleration on both source bucket and destination bucket will fix the problem

Additional Information/Context

Both my bucket in same AZ, I did not want to use Transfer acceleration.

The bucket created on today (2023/11/15) has this issue, if the source bucket and destination bucket are both created on 2023 April won't have this issue.

CLI version used

aws-cli / 2.13.34

Environment details (OS name and version, etc.)

AWS CloudShell

[h0806449f]

aws s3 ls s3://test-bucket-for-sync-source
will get same error and ask for turn on S3 Transfer Acceleration.

[RyanFitzSimmonsAK]

Hi @h0806449f, thanks for reaching out. Could you run aws configure get s3_use_accelerate_endpoint? If this value is true, then the CLI is trying to use S3 Transfer Acceleration endpoints for all S3 requests, even on buckets that don't have it enabled. You can change this value to false with aws configure set s3_use_accelerate_endpoint false. Hope that helps!

Relevant documentation : https://docs.aws.amazon.com/AmazonS3/latest/userguide/transfer-acceleration-examples.html

[h0806449f]

Thanks, problems solved.
Some instance behind AWS CloudShell set the .aws/config s3_use_accelerate_endpoint = true, some set this value as false

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[h0806449f]

Maybe the priority of default value and manual setting is wrong ?

cat ./.aws/config
[default]
s3 = use_accelerate_endpoint = true
s3_use_accelerate_endpoint = false

will cause
fatal error: An error occurred (InvalidRequest) when calling the ListObjectsV2 operation: S3 Transfer Acceleration is not configured on this bucket

Have to reset default value by using
aws configure set default.s3.use_accelerate_endpoint false
can fix the fatal error

[RyanFitzSimmonsAK]

Adding default. is specifying that you want to change these settings for the profile named default.

s3 = use_accelerate_endpoint = true is under the s3 key, and can be modified with the syntax aws configure set s3.use_accelerate_endpoint false. More information can be found here.

s3_use_accelerate_endpoint = false is under the name of the profile, and can be modified with the syntax aws configure set s3_use_accelerate_endpoint false.

It looks like the s3 key takes priority if both are set.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.