You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There are several issues with generating presigned url which do not fit in the title of this issue. So, let me try to explain my scenario and then write down all the issues.
Backend service in kubernetes cluster is uploading images to minio. Web clients (outside of the cluster) are getting presigned urls for those images to display them to the user. Address of minio is not the same when accessed by backend service and web clients since they run in different network environments. It means that when backend service generates presigned urls for the web client, those urls will not be accessible outside of the cluster, because MinioClient is using endpoint that was set during MinioClient building to generate those urls.
To overcome this, I create separate MinioClient instance with external endpoint address just for generating presigned urls. Issue is that even though in several issues here it was mentioned that generating presigned urls is client only operation, it is really not. If bucket that I'm trying to generate presigned url for does not exist in BucketRegionCache, PresignedGetObjectAsync will try to retrieve bucket region from minio server and it will fail miserably because MinioClient now has external endpoint address that is not accessible by backend service. We can argue that it is still generated on the client side, but if it needs the information from minio server to generate the url than it is really not.
Proposed band-aid solution
As far as I know region information does not need to be in presigned url, so just skip requesting bucket region from minio server. Then PresignedGetObjectAsync does not need to be an async operation anymore as one would expect for client only operation.
Proposed proper solution
Along with not including region in presigned url, generating presigned urls should allow using separate endpoint address, so that we don't need to create separate instance of MiniClient just to generate presigned urls with desired endpoint.
Also, BucketRegionCache being a static object is a design issue. If by any chance you would need to access two separate minio servers, two MinioClient instances would share that same cache although they really shouldn't. Static objects are evil. But in this case, if we take region out of the equation, than that becomes a separate issue.
Workaround
Apart from using separate MinioClient instance with the desired endpoint, this is required before generating presigned url:
if (!BucketRegionCache.Instance.Exists(bucketName))
BucketRegionCache.Instance.Add(bucketName, "us-east-1");
This will prevent PresignedGetObjectAsync code to try to retrieve region name from the minio server. Even though this works, it should not be considered as permanent solution since it relies on how the internals are implemented, which can change at any time, which would render this a non-working workaround.
The text was updated successfully, but these errors were encountered:
@ante-maric
Thank you for your suggestion.
I'll discuss it with the team as this might also impact all other SDKs.
I'll share it here as soon as a decision is made.
There are several issues with generating presigned url which do not fit in the title of this issue. So, let me try to explain my scenario and then write down all the issues.
Backend service in kubernetes cluster is uploading images to minio. Web clients (outside of the cluster) are getting presigned urls for those images to display them to the user. Address of minio is not the same when accessed by backend service and web clients since they run in different network environments. It means that when backend service generates presigned urls for the web client, those urls will not be accessible outside of the cluster, because MinioClient is using endpoint that was set during MinioClient building to generate those urls.
To overcome this, I create separate MinioClient instance with external endpoint address just for generating presigned urls. Issue is that even though in several issues here it was mentioned that generating presigned urls is client only operation, it is really not. If bucket that I'm trying to generate presigned url for does not exist in BucketRegionCache, PresignedGetObjectAsync will try to retrieve bucket region from minio server and it will fail miserably because MinioClient now has external endpoint address that is not accessible by backend service. We can argue that it is still generated on the client side, but if it needs the information from minio server to generate the url than it is really not.
Proposed band-aid solution
As far as I know region information does not need to be in presigned url, so just skip requesting bucket region from minio server. Then PresignedGetObjectAsync does not need to be an async operation anymore as one would expect for client only operation.
Proposed proper solution
Along with not including region in presigned url, generating presigned urls should allow using separate endpoint address, so that we don't need to create separate instance of MiniClient just to generate presigned urls with desired endpoint.
Also, BucketRegionCache being a static object is a design issue. If by any chance you would need to access two separate minio servers, two MinioClient instances would share that same cache although they really shouldn't. Static objects are evil. But in this case, if we take region out of the equation, than that becomes a separate issue.
Workaround
Apart from using separate MinioClient instance with the desired endpoint, this is required before generating presigned url:
This will prevent PresignedGetObjectAsync code to try to retrieve region name from the minio server. Even though this works, it should not be considered as permanent solution since it relies on how the internals are implemented, which can change at any time, which would render this a non-working workaround.
The text was updated successfully, but these errors were encountered: