Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bucket retention policy for sentinel-input-files bucket. #294

Open
sharkinsspatial opened this issue Jun 7, 2024 · 4 comments
Open
Assignees
Labels
enhancement New feature or request

Comments

@sharkinsspatial
Copy link
Collaborator

As completed in #281 the sentinel-input-files bucket should also have a retention policy. For this bucket it should be 6 months.

@sharkinsspatial sharkinsspatial added the enhancement New feature or request label Jun 7, 2024
@igaurab

This comment was marked as spam.

@chuckwondo
Copy link
Collaborator

@sharkinsspatial, shall we set the retention to 30 days for the mcp-dev-viirs env and 6 months for the mcp-production-viirs env?

cc: @madhuksridhar

@chuckwondo
Copy link
Collaborator

I have manually created these lifecycle rules:

  • hls-mcp-development-viirs-sentinel-input-files: expire objects after 15 days
  • hls-mcp-production-viirs-sentinel-input-files: expire objects after 120 days (we can dial it down further in January)

I'll add them to the CDK stack after the holidays.

@madhuksridhar
Copy link
Collaborator

@chuckwondo
I wanted to add a note here that since these buckets have replication rules enabled, both these actions need to be checked while creating the lifecycle rule

  • Expire current versions of objects
  • Permanently delete noncurrent versions of objects

The first one (expire) simply creates a delete marker but will not delete the noncurrent objects.
I edited the lifecycle rules with this change for the buckets listed above and we saw the expected drop in bucket volume over last few days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants