[bug]: external s3/minio #1318

MyWay · 2023-06-17T10:39:14Z

Is there an existing issue for this?

I have searched the existing issues

Current behavior

When I upload the logo, the loader is stuck, while on the console I get: "Something went wrong please try again later".

Steps to reproduce

In my docker-compose.yml I disabled plane-minio, createbuckets and set USE_MINIO to 0.
In my environment variables I did set AWS_S3_ENDPOINT_URL=s3.eu-west-2.wasabisys.com, AWS_REGION=eu-west-2 and all AWS related variables

Couldn't find any useful info neither on the docs, nor here among the issues.

Browser

Google Chrome

Version

Self-hosted

pablohashescobar · 2023-06-19T03:20:45Z

Hi @MyWay,

I noticed that you're encountering an issue with the endpoint URL. To resolve this, could you please try prepending https:// before the URL? This should ensure that the URL is using the correct protocol for the request.

MyWay · 2023-06-19T10:09:40Z

Hi @MyWay,

I noticed that you're encountering an issue with the endpoint URL. To resolve this, could you please try prepending https:// before the URL? This should ensure that the URL is using the correct protocol for the request.

Hi, i have tried your suggestion, but I'm getting the same error.

anvy2 · 2023-07-12T20:33:20Z

Any update or solution for this. I'm having the same issue. I've even tried minio setup when s3 didn't work thinking there might be some issue with the s3 endpoint url but no luck. Still can't make uploads work, the upload api gives 400 with the message "Something went wrong please try again later", that's it. I guess one good way is to log the actual error in the console or the logger so it would be easier to debug and know where it is failing than just ignoring the error with such a broad exception catch. I'd be happy to open a PR regarding the same

orbatschow · 2023-07-17T20:05:20Z

Maybe this is helpful:

I had the same problem and debugging was pretty hard. It turned out, that my self hosted minio setup had TLS verification enabled. Those were custom signed TLS certificates and plane was not able to verify the certificate correctly. I completely disable the minio TLS setup and configured plane to use plain HTTP. After this I was able to upload assets.

shibco · 2023-08-02T17:43:02Z

I have this issue too. I'm using OpenStack / Swift and I just cannot get anything to upload.

In the browser:

POST
https://[redacted]/api/users/file-assets/
[HTTP/2 400 Bad Request 992ms]
Object { error: "Something went wrong please try again later" }

In Plane's logs:

planebackend     | POST - /api/users/file-assets/ of Queries: 1
planebackend     | Bad Request: /api/users/file-assets/
planebackend     | 172.23.0.2:39992 - "POST /api/users/file-assets/ HTTP/1.1" 400

My env:

AWS_REGION="us-east-1" // Set by my Openstack provider
AWS_ACCESS_KEY_ID="[redacted]"
AWS_SECRET_ACCESS_KEY="[redacted]"
AWS_S3_ENDPOINT_URL="https://s3.[redacted]/object/v1/AUTH_[redacted]" // As per my Openstack provider's documentation
AWS_S3_BUCKET_NAME="plane-uploads"
FILE_SIZE_LIMIT=5242880

I'm not even sure how to diagnose this, there's nothing helpful at all in the logs.

shibco · 2023-08-03T15:56:47Z

A small update, issue not fully fixed however. In my Openstack case changing AWS_S3_ENDPOINT_URL="https://s3.[redacted]/object/v1/AUTH_[redacted]" to AWS_S3_ENDPOINT_URL="https://s3.[redacted]" allows for authentication and successful POST requests (eg file uploads.) However, actual bucket location is still at https://s3.[redacted]/object/v1/AUTH_[redacted]/plane-uploads/, which is not reflected in the AWS_S3_ENDPOINT_URL. This mismatch means the uploads can not be retrieved.

ka6u7ack · 2023-10-12T10:15:05Z

I solved it. My situation is more like django-storages issue.

I print the exception. The error message is An error occurred (AccessControlListNotSupported) when calling the PutObject operation: The bucket does not allow ACLs.
Make sure AWS_S3_ENDPOINT_URL="https://s3.amazonaws.com", not AWS_S3_ENDPOINT_URL="https://[bucket_name].s3.amazonaws.com"

Edit apiserver/plane/settings/production.py

Make sure AWS_S3_BUCKET_AUTH = True. It is important.

You need edit AWS_S3_ADDRESSING_STYLE. Based on aws region, you can refer to

Region	Setting
us-east-1	default or AWS_S3_SIGNATURE_VERSION = "s3v4"
us-east-2	AWS_S3_ADDRESSING_STYLE = "virtual"
ap-northeast-1	AWS_S3_SIGNATURE_VERSION = "s3v4"
ap-southeast-2	AWS_S3_ADDRESSING_STYLE = "virtual"
ap-south-1	AWS_S3_ADDRESSING_STYLE = "virtual"
eu-central-1	AWS_S3_ADDRESSING_STYLE = "virtual"
eu-central-1	AWS_S3_ADDRESSING_STYLE = "virtual"
eu-west-1	AWS_S3_SIGNATURE_VERSION = "s3v4" AWS_S3_ADDRESSING_STYLE = "virtual"
eu-west-2	AWS_S3_ADDRESSING_STYLE = "virtual"
eu-west-3	AWS_S3_ADDRESSING_STYLE = "virtual"
ca-central-1	AWS_S3_ADDRESSING_STYLE = "virtual"

My aws region is us-east-2. I replaced AWS_S3_ADDRESSING_STYLE = "auto" with AWS_S3_ADDRESSING_STYLE = "virtual".

(optional) AWS_S3_MAX_AGE_SECONDS = 7 * 24 * 60 * 60
(optional) Set AWS_S3_PUBLIC_URL, if your bucket has a CDN link. (Although the comment say This setting cannot be used with "AWS_S3_BUCKET_AUTH", actually it works.)

pablohashescobar · 2023-12-15T06:07:19Z

Hey @MyWay, can you check by upgrading to the latest version. This issue should be fixed now. Let us know if you are still facing this issue.

MyWay · 2023-12-15T07:41:20Z

Hey @MyWay, can you check by upgrading to the latest version. This issue should be fixed now. Let us know if you are still facing this issue.

How should I configure it for an alternative s3 in the latest version? Because upload is working, I can see the file in the bucket, but then the url generated by plane is pointing to my plane instance, so maybe I'm missing some setting.

pablohashescobar · 2023-12-18T19:02:34Z

You can turn the USE_MINIO environment to 0.

MyWay · 2023-12-18T19:09:32Z

You can turn the USE_MINIO environment to 0.

I did set it at 0, the file is uploaded, the filename is correct, but the url is mydomain.ext/mybucketname instead of my external s3 compatible service.

pablohashescobar · 2023-12-18T19:12:29Z

Oh okay @MyWay, can you also remove the AWS_S3_ENDPOINT_URL.

MyWay · 2023-12-18T19:14:50Z

Oh okay @MyWay, can you also remove the AWS_S3_ENDPOINT_URL.

If I do won't it use Amazon s3? 🤔

pablohashescobar · 2023-12-18T19:15:36Z

It will be still using Amazon S3 only.

MyWay · 2023-12-18T19:17:02Z

It will be still using Amazon S3 only.

But I'd like to use my s3 compatible provider.

pablohashescobar · 2023-12-18T19:19:28Z

Oh okay currently Plane only supports S3 and Minio what storage are you using @MyWay ?

MyWay · 2023-12-18T19:21:36Z

Oh okay currently Plane only supports S3 and Minio what storage are you using @MyWay ?

I see, that's the issue with my instance then. I have tried both wasabi and synology, currently on synology.

Apparently only missing thing is to generate the correct url using AWS_S3_ENDPOINT_URL.

pablohashescobar · 2023-12-19T17:49:37Z

@MyWay, you may have to also update the nginx.conf to point to your file server.

MyWay · 2023-12-19T18:04:12Z

@MyWay, you may have to also update the nginx.conf to point to your file server.

You mean I should rewrite urls to reflect my s3 instance, instead of expecting plane to do it?

pablohashescobar · 2023-12-19T19:01:24Z

You may have to do some changes the current plane setup either connects with S3 or minio for fetching assets for other services you may have to change the configuration a bit to make it work.

MyWay · 2023-12-19T19:45:57Z

I see. Since all of them implement the same API, do you plan to add support to any of the s3 services?

dbtek · 2023-12-29T11:29:27Z

Please see #3278 for using custom s3 endpoints

MyWay · 2023-12-31T10:36:38Z

Please see #3278 for using custom s3 endpoints

It's still not working for custom s3 endpoints from what I see.

amalpaul54111 · 2024-01-09T13:44:55Z

Couldn't upload files to AWS S3, I am using the script to run it on a linux server with docker.
I did update the environment variables with the working AWS credentials.

DATA STORE SETTINGS

USE_MINIO=0
AWS_REGION="eu-west-1"
AWS_ACCESS_KEY_ID="[reducted]"
AWS_SECRET_ACCESS_KEY="[reducted]"
AWS_S3_ENDPOINT_URL="https://s3.amazonaws.com"
AWS_S3_BUCKET_NAME="[reducted]"
FILE_SIZE_LIMIT=52428800

Is there some other change we have to make to enable it to upload to S3.

Getting the following error from plane-app-api-1

/api/workspaces/internal/file-assets/ HTTP/1.0" 400

truongnat · 2024-02-23T03:09:18Z

I facing same issue when setup default minio.

Bellow my config

# DATA STORE SETTINGS
USE_MINIO=1
AWS_REGION=""
AWS_ACCESS_KEY_ID=""
AWS_SECRET_ACCESS_KEY=""
AWS_S3_ENDPOINT_URL=<my-url>
AWS_S3_BUCKET_NAME=uploads
MINIO_ROOT_USER="<my-user>"
MINIO_ROOT_PASSWORD="<my-password>"
BUCKET_NAME=uploads
FILE_SIZE_LIMIT=5242880

But in UI, i see it call api workspace/<name>/file-assets and it return 500 server error (Something went wrong please try again later)

Please help

srinivaspendem · 2024-05-17T06:45:37Z

This issue will be resolved when you update to the latest version of Plane.

MyWay · 2024-05-17T19:50:05Z

I'm already using latest stable, which version you are referring exactly?

RaylenZed · 2024-06-24T06:13:21Z

I'm already using latest stable, which version you are referring exactly?

@MyWay

I'm using AWS s3 , started with 500 error as well, then when I recreated the bucket, I enabled ACLs and it's working fine now, so you can test it from that.

Translated from DeepL

MyWay added the 🐛bug Something isn't working label Jun 17, 2023

rhea0110 added 🛫plane Sync issues onto Plane 🏠self-hosted labels Jun 19, 2023

vihar assigned pablohashescobar and mguptahub Dec 14, 2023

vihar added the 🔥urgent label Dec 14, 2023

srinivaspendem closed this as completed May 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bug]: external s3/minio #1318

[bug]: external s3/minio #1318

MyWay commented Jun 17, 2023

pablohashescobar commented Jun 19, 2023 •

edited

Loading

MyWay commented Jun 19, 2023

anvy2 commented Jul 12, 2023

orbatschow commented Jul 17, 2023

shibco commented Aug 2, 2023 •

edited

Loading

shibco commented Aug 3, 2023

ka6u7ack commented Oct 12, 2023 •

edited

Loading

pablohashescobar commented Dec 15, 2023

MyWay commented Dec 15, 2023

pablohashescobar commented Dec 18, 2023 •

edited

Loading

MyWay commented Dec 18, 2023

pablohashescobar commented Dec 18, 2023

MyWay commented Dec 18, 2023 •

edited

Loading

pablohashescobar commented Dec 18, 2023

MyWay commented Dec 18, 2023

pablohashescobar commented Dec 18, 2023

MyWay commented Dec 18, 2023 •

edited

Loading

pablohashescobar commented Dec 19, 2023

MyWay commented Dec 19, 2023

pablohashescobar commented Dec 19, 2023

MyWay commented Dec 19, 2023

dbtek commented Dec 29, 2023

MyWay commented Dec 31, 2023

amalpaul54111 commented Jan 9, 2024 •

edited

Loading

truongnat commented Feb 23, 2024

srinivaspendem commented May 17, 2024 •

edited

Loading

MyWay commented May 17, 2024

RaylenZed commented Jun 24, 2024

[bug]: external s3/minio #1318

[bug]: external s3/minio #1318

Comments

MyWay commented Jun 17, 2023

Is there an existing issue for this?

Current behavior

Steps to reproduce

Browser

Version

pablohashescobar commented Jun 19, 2023 • edited Loading

MyWay commented Jun 19, 2023

anvy2 commented Jul 12, 2023

orbatschow commented Jul 17, 2023

shibco commented Aug 2, 2023 • edited Loading

shibco commented Aug 3, 2023

ka6u7ack commented Oct 12, 2023 • edited Loading

pablohashescobar commented Dec 15, 2023

MyWay commented Dec 15, 2023

pablohashescobar commented Dec 18, 2023 • edited Loading

MyWay commented Dec 18, 2023

pablohashescobar commented Dec 18, 2023

MyWay commented Dec 18, 2023 • edited Loading

pablohashescobar commented Dec 18, 2023

MyWay commented Dec 18, 2023

pablohashescobar commented Dec 18, 2023

MyWay commented Dec 18, 2023 • edited Loading

pablohashescobar commented Dec 19, 2023

MyWay commented Dec 19, 2023

pablohashescobar commented Dec 19, 2023

MyWay commented Dec 19, 2023

dbtek commented Dec 29, 2023

MyWay commented Dec 31, 2023

amalpaul54111 commented Jan 9, 2024 • edited Loading

DATA STORE SETTINGS

truongnat commented Feb 23, 2024

srinivaspendem commented May 17, 2024 • edited Loading

MyWay commented May 17, 2024

RaylenZed commented Jun 24, 2024

pablohashescobar commented Jun 19, 2023 •

edited

Loading

shibco commented Aug 2, 2023 •

edited

Loading

ka6u7ack commented Oct 12, 2023 •

edited

Loading

pablohashescobar commented Dec 18, 2023 •

edited

Loading

MyWay commented Dec 18, 2023 •

edited

Loading

MyWay commented Dec 18, 2023 •

edited

Loading

amalpaul54111 commented Jan 9, 2024 •

edited

Loading

srinivaspendem commented May 17, 2024 •

edited

Loading