You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Virtual-host-style links move the bucket name in the domain name. With our service, that might look like: https://shroff-public.s3proxy.janelia.org/shroff_2022...6x6x6nm_2xbin.n5/
When your URIs are not "official" AWS S3 URIs, there is no way to determine which style is being used from the URI alone. Some kind of test requests would need to be sent to determine the style. Another possibility is to enforce the <bucketname>.s3. convention that AWS uses internally, and parse the URI in a similar way to how they do it. It's not clear how compatible that would be with other services, but we could easily implement it in Janelia's S3 proxy.
The AWS SDK for Java doesn't make it super intuitive to work with virtual-host-style links for non-AWS endpoints, but it does seem possible. For example, if if your endpoint is https://shroff-public.s3.janelia.org then you just need to use an empty bucket name to list the contents of the bucket: listObjectsV2("")
Since non-AWS endpoints are a separate path in the code, it's not likely that they would be affected even if AWS deprecates path-style URIs for the official S3 service. Therefore, I would consider this feature a low priority from the future-proofing standpoint.
The text was updated successfully, but these errors were encountered:
We have an S3 proxy service which emulates the S3 protocol, and serves links which work in N5 Viewer, e.g. https://s3proxy.janelia.org/shroff-public/shroff...2xbin.n5/
@axtimwalde asked me to look into supporting virtual-host-style links, since AWS has indicated that it may deprecated path-style links in the future.
Virtual-host-style links move the bucket name in the domain name. With our service, that might look like:
https://shroff-public.s3proxy.janelia.org/shroff_2022...6x6x6nm_2xbin.n5/
I implemented it in my dev environment but it doesn't work in N5 Viewer because some places in the code assume path-style links when the endpoint is not AWS:
https://github.com/saalfeldlab/n5-aws-s3/blob/master/src/main/java/org/janelia/saalfeldlab/n5/s3/AmazonS3Utils.java#L50
https://github.com/saalfeldlab/n5-aws-s3/blob/master/src/main/java/org/janelia/saalfeldlab/n5/s3/AmazonS3Utils.java#L137
A few more observations:
<bucketname>.s3.
convention that AWS uses internally, and parse the URI in a similar way to how they do it. It's not clear how compatible that would be with other services, but we could easily implement it in Janelia's S3 proxy.https://shroff-public.s3.janelia.org
then you just need to use an empty bucket name to list the contents of the bucket:listObjectsV2("")
The text was updated successfully, but these errors were encountered: