Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Max pending/unpinned quota #1870

Closed
alanshaw opened this issue Apr 29, 2022 · 2 comments · May be fixed by #1974
Closed

Max pending/unpinned quota #1870

alanshaw opened this issue Apr 29, 2022 · 2 comments · May be fixed by #1974
Labels
kind/enhancement A net-new feature or improvement to an existing feature P1 High: Likely tackled by core team if no one steps up potential/support-pi

Comments

@alanshaw
Copy link
Contributor

We've recently had a user pin hundreds of thousands of CIDs that they do not have access to and can never retrieve.

This adds load to our cluster and dagcargo that is currently never resolved.

The idea here is to add a quota for maximum "in flight" pins. This effectively puts a per user cap on the number of pins that can be submitted that never become pinned. The user then starts to receive 4xx errors for pin requests, which should prompt them to delete bad pins. It also serves as a crude rate limiting mechanism.

Alternatively an adequete cleanup job could take care of this, but the idea here is to reduce that load by encouraging users to clean up after themselves.
@alanshaw alanshaw added kind/enhancement A net-new feature or improvement to an existing feature need/triage Needs initial labeling and prioritization labels Apr 29, 2022
@ribasushi
Copy link
Contributor

Note - this should either be limited to PSA only, or at least have different PSA / non-PSA limits.
The PSA ones are prone to effectively-abuse, while the upload-based ones are somewhat symmetric.

@mbommerez mbommerez mentioned this issue May 3, 2022
Open
2 tasks
@dchoi27 dchoi27 added P1 High: Likely tackled by core team if no one steps up and removed need/triage Needs initial labeling and prioritization labels May 3, 2022
@vasco-santos
Copy link
Contributor

Looking into this, I think the best approach is to add a route middleware to routes POST /pins and /pins/:requestId, augmenting the authorization layer to also check the on going requests. We can have ENV variable with max amount

@vasco-santos vasco-santos mentioned this issue Jun 9, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/enhancement A net-new feature or improvement to an existing feature P1 High: Likely tackled by core team if no one steps up potential/support-pi
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: restrict in flight psa requests with max quota nftstorage/nft.storage
5 participants
@alanshaw @ribasushi @vasco-santos @dchoi27 @elizabeth-griffiths