feat(aws_s3): Add S3 compatibility environment variables for non-AWS services

[jovermier]

Summary

• Add environment variables to disable AWS-specific S3 headers for compatibility with non-AWS S3 services like MinIO and Rook Ceph RGW
• AWS_S3_DISABLE_SSE=true disables server-side encryption headers (x-amz-server-side-encryption)
• AWS_S3_DISABLE_CHECKSUMS=true disables checksum algorithm headers (x-amz-checksum-algorithm)

Problem

The current Convex backend hard-codes AWS-specific headers in S3 multipart upload operations:

• ServerSideEncryption::Aes256
• ChecksumAlgorithm::Crc32

These headers cause compatibility issues with S3-compatible services that don't support AWS-specific features, leading to multipart upload failures.

Solution

• Added two new environment variables in aws_utils crate for S3 compatibility configuration
• Modified aws_s3 storage implementation to conditionally add AWS headers based on environment variables
• Maintains backward compatibility (headers are added by default when variables are not set)

Files Changed

• crates/aws_utils/src/lib.rs: Added environment variable configuration and accessor functions
• crates/aws_s3/src/storage.rs: Updated multipart upload methods to conditionally add headers

Testing

• Code compiles successfully with all dependencies
• Backward compatibility maintained (default behavior unchanged)
• Ready for testing with MinIO and other S3-compatible services

Benefits

• Enables self-hosted Convex to work with a wider range of S3-compatible storage backends
• Improves compatibility with MinIO, Rook Ceph RGW, and other S3 implementations
• Maintains full backward compatibility with existing AWS S3 deployments

🤖 Generated with Claude Code

[Copilot]

Pull Request Overview

This PR adds environment variables to disable AWS-specific S3 headers for compatibility with non-AWS S3 services like MinIO and Rook Ceph RGW.

• Added AWS_S3_DISABLE_SSE and AWS_S3_DISABLE_CHECKSUMS environment variables to control AWS-specific header inclusion
• Modified S3 multipart upload operations to conditionally add server-side encryption and checksum headers
• Maintains backward compatibility by defaulting to include headers when environment variables are not set

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
crates/aws_utils/src/lib.rs	Added environment variable configuration and accessor functions for S3 compatibility settings
crates/aws_s3/src/storage.rs	Updated multipart upload methods to conditionally add AWS-specific headers based on environment variables

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[jovermier]

Thanks for the code review feedback! ✅

Issue Addressed: The logic duplication between start_upload and start_client_driven_upload methods.

Solution Implemented:

1. Extracted helper method: Created configure_multipart_upload_builder() in the S3Storage impl block
2. Eliminated duplication: Both methods now use the same helper for AWS header configuration
3. Preserved domain knowledge: Kept the original authors' important technical comment about multipart upload checksum algorithms
4. Maintained functionality: Identical behavior with improved maintainability

Commit: 423c57e - refactor(aws_s3): extract helper method to reduce code duplication

Future changes to the AWS header logic now only need to be made in one place! The code is cleaner and more maintainable while preserving all the original functionality and documentation.

[nipunn1313]

you'll also want to add them to self-hosted/docker/docker-compose.yml

[nipunn1313]

I'll take care of it

[jovermier]

Alright. Let me know if there is anything else you want me to fix or change.

[nipunn1313]

@gautamg795 - can you review? I took a look and it seems reasonable to me, but would appreciate another set of eyes.

Would solve #85

[gautamg795]

lgtm, pretty clean solution

[nipunn1313]

merged in 683d4bd

[nipunn1313]

Thanks @jovermier for the contribution!