Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PresignedGetObjectAsync should be client only operation, but it is not really #861

Open
ante-maric opened this issue Sep 6, 2023 · 1 comment
Assignees

Comments

@ante-maric
Copy link

There are several issues with generating presigned url which do not fit in the title of this issue. So, let me try to explain my scenario and then write down all the issues.

Backend service in kubernetes cluster is uploading images to minio. Web clients (outside of the cluster) are getting presigned urls for those images to display them to the user. Address of minio is not the same when accessed by backend service and web clients since they run in different network environments. It means that when backend service generates presigned urls for the web client, those urls will not be accessible outside of the cluster, because MinioClient is using endpoint that was set during MinioClient building to generate those urls.

To overcome this, I create separate MinioClient instance with external endpoint address just for generating presigned urls. Issue is that even though in several issues here it was mentioned that generating presigned urls is client only operation, it is really not. If bucket that I'm trying to generate presigned url for does not exist in BucketRegionCache, PresignedGetObjectAsync will try to retrieve bucket region from minio server and it will fail miserably because MinioClient now has external endpoint address that is not accessible by backend service. We can argue that it is still generated on the client side, but if it needs the information from minio server to generate the url than it is really not.

Proposed band-aid solution
As far as I know region information does not need to be in presigned url, so just skip requesting bucket region from minio server. Then PresignedGetObjectAsync does not need to be an async operation anymore as one would expect for client only operation.

Proposed proper solution
Along with not including region in presigned url, generating presigned urls should allow using separate endpoint address, so that we don't need to create separate instance of MiniClient just to generate presigned urls with desired endpoint.
Also, BucketRegionCache being a static object is a design issue. If by any chance you would need to access two separate minio servers, two MinioClient instances would share that same cache although they really shouldn't. Static objects are evil. But in this case, if we take region out of the equation, than that becomes a separate issue.

Workaround
Apart from using separate MinioClient instance with the desired endpoint, this is required before generating presigned url:

        if (!BucketRegionCache.Instance.Exists(bucketName))
            BucketRegionCache.Instance.Add(bucketName, "us-east-1");

This will prevent PresignedGetObjectAsync code to try to retrieve region name from the minio server. Even though this works, it should not be considered as permanent solution since it relies on how the internals are implemented, which can change at any time, which would render this a non-working workaround.

@ebozduman
Copy link
Collaborator

@ante-maric
Thank you for your suggestion.
I'll discuss it with the team as this might also impact all other SDKs.
I'll share it here as soon as a decision is made.

@ebozduman ebozduman self-assigned this Sep 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants