This repository has been archived by the owner on Aug 14, 2024. It is now read-only.
docs(self-hosted): external storage configurations #1269
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| --- | ||
| title: External Storage | ||
| --- | ||
|
|
||
| In some cases, storing Sentry data on-disk is not really something people can do. Sometimes, it's better if they can offload it into some bucket storage (like AWS S3 or Google Cloud Storage). | ||
|
There was a problem hiding this comment. This seems a bit confusing, but I think adding another page later after this about "Unsupported Workflows" in which we can specify more about what kind of things that we can't offer support to (external Redis, external Postgres, installing third party plugins for extending some stuff). See @azaslavsky's comment here #1269 (comment) |
||
|
|
||
| <Alert title="Note" level="info"> | ||
| After changing configuration files, re-run the <code>./install.sh</code> script, to rebuild and restart the containers. See the <Link to="/self-hosted/#configuration">configuration section</Link> for more information. | ||
| </Alert> | ||
|
|
||
| ## Sentry | ||
|
|
||
| The Sentry service has a abstraction called "filestore" that handles storing attachment, sourcemap (release artifacts), and replays. Filestore configuration for Sentry should be configured on the `sentry/config.yml` file. | ||
|
|
||
| ### Google Cloud Storage backend | ||
|
|
||
| The configuration for GCS backend is pointed to `sentry.filestore.gcs.GoogleCloudStorage`. You will need to set `GOOGLE_APPLICATION_CREDENTIALS` environment variable. For more information, refer to the [Google Cloud documentation for setting up authentication](https://cloud.google.com/storage/docs/reference/libraries#setting_up_authentication). | ||
|
|
||
| ```yaml | ||
| filestore.backend: "gcs" | ||
| filestore.options: | ||
| bucket_name: "..." | ||
| ``` | ||
|
|
||
| ### S3 backend | ||
|
There was a problem hiding this comment. Just to make sure, have you tried the Azure/s3 compatible backend without issues? We're using GCS so wanted to make sure There was a problem hiding this comment. Yes! See my patch getsentry/self-hosted@master...teknologi-umum:sentry:teknum-patch |
||
|
|
||
| <Alert title="Note" level="warning"> | ||
| Although S3 support is available, it is not thoroughly tested and being used by Sentry SaaS internally. Therefore, it is not something that Sentry folks will offer very good support for it. | ||
| </Alert> | ||
|
|
||
| The configuration for S3-compatible backend is pointed to `sentry.filestore.s3.S3Boto3Storage`. | ||
|
|
||
| ```yaml | ||
| filestore.backend: 's3' | ||
| filestore.options: | ||
| bucket_acl: 'private' | ||
| default_acl: 'private' | ||
| access_key: '<REDACTED>' | ||
| secret_key: '<REDACTED>' | ||
| bucket_name: 'my-bucket' | ||
| region_name: 'auto' | ||
| endpoint_url: 'https://<REDACTED>' # If you're not using AWS. | ||
| addressing_style: 'path' # For regular AWS S3, use "auto" or "virtual". For other S3-compatible API like MinIO or Ceph, use "path". | ||
| signature_version: 's3v4' | ||
| ``` | ||
|
|
||
| Refer to [botocore configuration](https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html) for valid configuration values. | ||
|
|
||
| ## Vroom | ||
|
|
||
| Vroom is the service that handles profiling. By default the data for profiling is saved on local filesystem. On self-hosted deployment, this should be done by overriding the `SENTRY_BUCKET_PROFILES` environment variable. It's also possible that additional environment variables should be added, depending on the backend of choice. | ||
|
|
||
| ### Google Cloud Storage backend | ||
|
|
||
| You will need to set `GOOGLE_APPLICATION_CREDENTIALS` environment variable. For more information, refer to the [Google Cloud documentation for setting up authentication](https://cloud.google.com/storage/docs/reference/libraries#setting_up_authentication). | ||
|
|
||
| ```bash | ||
| gs://my-bucket | ||
| ``` | ||
|
Comment on lines
+53
to
+59
There was a problem hiding this comment. I haven't tested this. I only know how to configure this. Can you guys test this out on your dogfood instance? |
||
|
|
||
| ### S3 backend | ||
|
|
||
| <Alert title="Note" level="warning"> | ||
| Although S3 support is available, it is not thoroughly tested and being used by Sentry SaaS internally. Therefore, it is not something that Sentry folks will offer very good support for it. | ||
| </Alert> | ||
|
|
||
| ```bash | ||
| # For regular AWS S3 | ||
| s3://my-bucket?awssdk=v1®ion=us-west-1&endpoint=amazonaws.com | ||
|
|
||
| # For other S3-compatible APIs | ||
| s3://my-bucket?awssdk=v1®ion=any-region&endpoint=minio.yourcompany.com&s3ForcePathStyle=true&disableSSL | ||
| ``` | ||
|
|
||
| Additional environment variables should be provided: | ||
| - `AWS_ACCESS_KEY=foobar` | ||
| - `AWS_SECRET_KEY=foobar` | ||
|
There was a problem hiding this comment. nit (here and elsewhere): change the value from |
||
| - `AWS_SESSION_TOKEN=foobar` (optional) | ||
|
|
||
| Further explanation on the query string options: | ||
| - `region`: The AWS region for requests. | ||
| - `endpoint`: The endpoint URL (hostname only or fully qualified URI). | ||
| - `disableSSL`: A value of "true" disables SSL when sending requests. | ||
| - `s3ForcePathStyle`: A value of "true" forces the request to use path-style addressing. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we should generalize to "Data Storage" or something.
This way this document can explain where the data is stored by default and can list alternatives if there are any.
Could have the following sections:
We should probably either rename those section to what the specifically store or explain that in the intro because "Vroom" is not very descriptive but if it's explained that that component is responsible for (ingest and) storing profiling data it makes a lot more sense.
Maybe with until someone else also chimes in before rewriting the whole thing in case I'm off base with this outline but this sounds like a document I would love to have had when I started my self-hosted adventures 👍
For the Object Storage thing we might want to link to the relevant documentation instead of adding examples for every option under the sun because otherwise there is no bound to the size of this document.