Feature/user rate limits #1342
Open
Feature/user rate limits #1342
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gecBurton
reviewed
Jan 29, 2025
| @classmethod | ||
| def get_since(cls, user: User, since: datetime) -> Sequence["ChatMessage"]: | ||
| """Returns all chat messages for a given user, with the most recent message after 'since'""" | ||
| return cls.objects.filter(chat__user=user, created_at__gt=since) |
There was a problem hiding this comment.
Suggested change
| return cls.objects.filter(chat__user=user, created_at__gt=since) | |
| cls.objects.filter(user=user).filter(created_at_gt=since).annotate(Sum("token_count"))["token_count__sum"] or 0 |
you can get the database to do the sum here?
There was a problem hiding this comment.
and this could now be moved to be a method belonging to the User model?
There was a problem hiding this comment.
but i think that there is a bigger problem here... the ChatMessage.tocken_count only includes what the user has typed, and not the inlcuded files, I think we need to make this change first?
gecBurton
approved these changes
Jan 30, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
Users can break each others sessions by consuming our token quota on backend models. We want users to only be able to consume their share of these quotas
Changes proposed in this pull request
Add a user rate limiter which imposes a max tokens per minute limit for all users across all models
Things to check