Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support path-style access URLs #66

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

elliot42
Copy link
Contributor

There are two ways to generate S3 object URLs, virtual-hosted style
URLs vs. path-style URLs 1:

  • Virtual-hosted-style: http://bucket.s3.amazonaws.com
  • Path-style: http://s3.amazonaws.com/bucket

Virtual-hosted-style is Amazon's preferred default, however HTTPS is
not compatible with all virtual-hosted-style URLs. Specifically,
virtual-hosted-style buckets with dots in their names always cause
HTTPS cert validation errors, as per RFC 2818 2:

https://foo.bar.s3.amazonaws.com/key

Path-style access works fine with HTTPS, without forcing a bucket rename:

https://s3.amazonaws.com/foo.bar/key

This commit allows configuring the client for path-style access via
the cred map:

(let [cred {:access-key ...
            :secret-key ...
            :path-style-access? true}]
  (generate-presigned-url cred bucket key))

Note that when using path style access you may need to manually specify your
region-specific S3 endpoint 1:

(let [cred {...
            :path-style-access? true
            :endpoint "s3-us-west-1.amazonaws.com"}]
  ...)

@weavejester
Copy link
Owner

As :path-style-access is a boolean, can you add a ? onto the end?

@elliot42 elliot42 force-pushed the path-style-access branch 2 times, most recently from f134bc8 to 66b959f Compare September 26, 2014 12:53
There are two ways to generate S3 object URLs, virtual-hosted style
URLs vs. path-style URLs [1]:

- Virtual-hosted-style: `http://bucket.s3.amazonaws.com`
- Path-style: `http://s3.amazonaws.com/bucket`

Virtual-hosted-style is Amazon's preferred default, however HTTPS is
not compatible with all virtual-hosted-style URLs.  Specifically,
virtual-hosted-style buckets with dots in their names always cause
HTTPS cert validation errors, as per RFC 2818 [2][3]:

    https://foo.bar.s3.amazonaws.com/key

Path-style access works fine with HTTPS, without forcing a bucket rename:

    https://s3.amazonaws.com/foo.bar/key

This commit allows configuring the client for path-style access via
the cred map:

    (let [cred {:access-key ...
                :secret-key ...
                :path-style-access? true}]
      (generate-presigned-url cred bucket key))

Note that when using path style access you may need to manually specify your
region-specific S3 endpoint [1]:

    (let [cred {...
                :path-style-access? true
                :endpoint "s3-us-west-1.amazonaws.com"}]
      ...)

[1]: http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html
[2]: http://www.ietf.org/rfc/rfc2818.txt
[3]: http://shlomoswidler.com/2009/08/amazon-s3-gotcha-using-virtual-host.html
@elliot42
Copy link
Contributor Author

Updated to include ? at ends of keyword keys/symbols.

@weavejester
Copy link
Owner

Thanks. Could you also slim down the commit message? There's no need to explain everything about the change. You can just provide a small explanation and a link to more details.

Add support for path-style access

This is necessary to support HTTPS URLs to buckets with periods in
their name.
See: http://shlomoswidler.com/2009/08/amazon-s3-gotcha-using-virtual-host.html

Conversely, it would be useful to add more information to the namespace docstring, detailing the option you've added. The docstring is currently a little cluttered, so you may want to reformat it like so:

  "Functions to access the Amazon S3 storage service.

  Each function takes a map of credentials as its first argument, which must
  contain the following keys:

    :access-key - the AWS access key ID
    :secret-key - the AWS secret access key

  The credentials may optionally have keys:

    :endpoint           - a URL or keyword refering to the AWS endpoint
    :proxy              - a map locating a HTTP proxy
    :path-style-access? - true if path-style access should be used

  The :proxy map must contain the following keys:

    :host - the hostname of the proxy
    :port - the port of the proxy

  And may contain:

    :user        - the user to use for the proxy
    :password    - the password of the user
    :domain      - the domain of the user, if applicable
    :workstation - the workstation of the user, if applicable"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants