Blocklist rewrite #11
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
blocklist.py contains the initial classes and code for the new blocklist system. The intention of this system is to unify blocking to one new mongodb collection and vastly simplify bot code required to manage it. Additionally, most simply methods should be side effect free.
migrations.py contains functions to migrate legacy blocklists to the new blocklist. contains handling for both legacy modmail string and the newer dict storage
Adds add_block, unblock_id, and is_id_blocked to blocklist class
✅ Deploy Preview for modmail-docs canceled.
|
khakers
force-pushed
the
blocklist-rewrite
branch
from
d91e813
to
9248c6e
Compare
khakers
force-pushed
the
blocklist-rewrite
branch
from
9248c6e
to
3a37a34
Compare
now with more DRY
is_id_blocked now returns a tuple[bool, Optional[BlocklistItem]] so the calling code can get information about the block easily
…ot.py block related code
…blocklist.is_user_blocked() instead of core is_blocked()
|
I've left in some of the legacy checking code that no core bot code uses for the moment, with logged warnings. It could be removed now or in the future, but I was a bit concerned about completely nuking some plugin that may be using it for whatever reason |
|
Thanks for the PR ( •̀ ω •́ )✧ |
raidensakura
added a commit
that referenced
this pull request
Dec 27, 2023
* A few optimizations - Reduce Docker image size - Migrate requirements.txt to Pipfile - Remove unused dependencies - Add a few unnecessary files in .dockerignore * Pull modmail-dev#3281 * Pull modmail-dev#3279 * Pull modmail-dev#3278 * Pull modmail-dev#3277 & proper indentation * Squashed commit of the following: commit 2e32d6e Merge: 51a023c b21037c Author: Martin <[email protected]> Date: Sun Apr 9 21:22:08 2023 +0200 Merge branch 'modmail-dev:master' into master commit 51a023c Author: Martin <[email protected]> Date: Mon Apr 3 13:35:56 2023 +0200 Changelog+Small fixes commit f5f25b5 Author: Martin <[email protected]> Date: Mon Apr 3 13:10:35 2023 +0200 confirm_thread_creation Buttons instead of reactions commit 11e094c Merge: ebd38d3 e919304 Author: Taku <[email protected]> Date: Tue Mar 14 05:19:54 2023 -0700 Merge branch 'master' into master commit ebd38d3 Author: Taku <[email protected]> Date: Tue Mar 14 05:18:17 2023 -0700 Fix indentations commit b77e435 Author: Martin <[email protected]> Date: Sun Mar 12 17:22:42 2023 +0100 Reminder Plugin Created a reminder plugin * Pull modmail-dev/modmail#modmail-dev#3271 * Squashed commit of the following: commit 13437cd Author: baptiste0928 <[email protected]> Date: Tue Mar 21 22:29:24 2023 +0100 Format with black commit 8a6ecd6 Merge: d9e364e 02250e8 Author: Taku <[email protected]> Date: Tue Mar 21 13:30:40 2023 -0700 Merge branch 'development' into feat/guild-icon-size Signed-off-by: Taku <[email protected]> commit d9e364e Author: baptiste0928 <[email protected]> Date: Sun Mar 19 20:17:35 2023 +0100 Update changelog commit d0bfeb4 Author: baptiste0928 <[email protected]> Date: Sun Mar 19 20:06:20 2023 +0100 Set size for guild icons on embeds * Format with black * Pull modmail-dev/modmail#modmail-dev#3233 * Squashed commit of the following: commit cf3444a Author: Khakers <[email protected]> Date: Tue Jan 24 14:13:15 2023 -0800 Add dm_channel_id field to log entry * Update changelog, workflow & remove bandit * Pull modmail-dev#3272 + lock dependencies * Add docs boilerplate + bumppython version * Dpy bump, add versions for ?about * Format with black * Update README.md * Fix permission error when installing plugin * Add new mention_message config, black + ruff * Switch from root to app user for Docker * Add support for trailing space in prefix * Add logviewer as built-in plugin * logviewer plugin enhancements * Minor logviewer changes * Update changelog + logviewer pomelo change * add pomelo for logbase * Change logviewer favicon * Refactor loglist object with proper serialization * Remove favicon to untrack later * fix log entry key error * Replace local favicon with remote avatar * Add markdown header support * Switch repo link and branch * Change more repo links * Handle properly when loglist is the index page * toggle off internal message checkbox on load * Remove trailing slash, fix broken log url on index * Fix broken default avatar url * Fix pathing issue on displaying log list in index * Add placeholder for not logged in user * Fix blockquote parsing emote ending * Whoops, remove comment * Fix blockquote parsing + oauth navbar indicator * Update docs * config and docs change * Update FUNDING.yml * Update docs * Update ignores + workflow * Tweak loadcheck * tweak loadcheck (2) * blegh * ✨ * Attempt to fix invalid guild_id on test * Docs update * Terminate process more gracefully * Update dependencies for Python 3.11 * Refactor logviewer pomelo changes * Minify docker image size * Remove unused files * bump black to fix checks * Update dev requirements * Update workflow * More pomelo changes * Replace avatar with local asset * Fix broken author avatar on `logs` command * Implement pomelo on channel name * Format with black * Add pre-commit hook * Update requirements * Overhaul block functionality * Add plugin uninstall alias * Update python badge * Update README.md * Update README.md * Fix block datetime not being timezone aware * Remove cairosvg * Fix copy paste mistake * Another copy paste issue sigh * Fix NoneType on parse * Improve block check and add message cooldown * bump dpy * Update internal dpy version * Update ?info command for pomelo * Bump version * Fix missing package error * Fix index compile error * Fix space in filepath failing to load local plugin * Escape codeblock language in class name * Fix genesis message retrieval failure * Update docs * Don't build image on PR * Blocklist rewrite (#11) * Add blocklist blocklist.py contains the initial classes and code for the new blocklist system. The intention of this system is to unify blocking to one new mongodb collection and vastly simplify bot code required to manage it. Additionally, most simply methods should be side effect free. * Add migrations.py migrations.py contains functions to migrate legacy blocklists to the new blocklist. contains handling for both legacy modmail string and the newer dict storage * Add blocklist bot.py integration * rename BlockedUser to BlocklistItem & add more blocklist methods Adds add_block, unblock_id, and is_id_blocked to blocklist class * migrate block and unblock commands to new blocklist * fix overzealous ide rename * add get_all_blocks method and documentation + cleanup to blocklist.py * refactor blocked command to use new block system * cleanup block command code now with more DRY * migrate bot.py is_blocked method to new blocklist system * migrate whitelist command and refactor is_id_blocked to return tuple is_id_blocked now returns a tuple[bool, Optional[BlocklistItem]] so the calling code can get information about the block easily * rename BlocklistItem to BlocklistEntry * Add deprecation warning logs to legacy block code and docstrings to bot.py block related code * clean up migrations.py code * Add [p]migration command * add final deprecation docstrings and refactor contact command to use blocklist.is_user_blocked() instead of core is_blocked() * remove blocklist print statements * Format with black --------- Co-authored-by: Raiden Sakura <[email protected]> * fix note code upgrade signature of note code state store_user method to match expected discord.py method use a typed checked dict for author instead of making a declaring a class in a for loop that replicates a different type. hopefully a type checker would raise warnings the next time one of these changes * Fix failing tests and docker build * fix blocklist bugs (#13) * fix missing await statements in new block method usage * remove Self type importing and usage * rename placeholder variable name * StrEnum is only in 3.11... --------- Co-authored-by: Raiden Sakura <[email protected]> * oops * Fix object is not subscriptable for 3.8 * Pull 2 changes from upstream (#14) * Fixed unset registry_plugins_only causing non-registry plugins to fail to install * To avoid confusion, config.get() only accepts one pos argument now --------- Co-authored-by: Taku <[email protected]> * Format with black * More blocklist fixes (#15) * update requirements.txt * fix blocklist migration detection * fix migrations.py type hints for python 3.8 * Add index for channel_id Channel_id is the main query field for nearly every log related database operation and for some reason it isn't indexed. This should speed up all log related database reliant operations especially on large collections * Update CHANGELOG.md Update changelog with my changes * Remove CairoSVG dependencies from Dockerfile * Improve is_image detection to actually detect images (excluding videos) and add content_type value * Fix logviewer positional arg error * partially reverts 2f76202 which added lottie sticker support and updates dependency lock (#20) * partially reverts 2f76202 which added lottie sticker support and updates dependency lock * Lint --------- Co-authored-by: Raiden Sakura <[email protected]> * Synchronize thread NSFW status and title with the database (#18) * Changing the thread title or nsfw status now immediately updates in the database !Changes signature of thread.set_title() adds update_nsfw and update_title methods to the mongodb client * Lint --------- Co-authored-by: Raiden Sakura <[email protected]> * Refactor docker workflow to multiplatform build (#21) Upgrades out of date action & adds dependency actions (QEMU & BuildX) for multiplatform build. Sets platforms to Linux/amd64 and Linux/arm64 * Update funding * Remove internal logviewer plugin * Add .gitignore for local plugin * Migrate to poetry * Update workflow for poetry * Update workflow * Oops, forgot to install deps * Update launch command * Attempt to solve image not building arm64 * Add missing libffi * Refactor Dockerfile * Resolve deprecated pkg_resources warning * Update changelog * Bump workflow versions * oops * Add [speed] extra for dpy * Bump dpy, don't exit when dpy version mismatch * Fix sticker replies not sending * Update changelog * Improved join/leave message for multiple servers. From modmail-dev@0086e04 * Fix code scanning alert #5 * Fixed compat with MissingRequiredArgument requiring additional param modmail-dev@9464c5d * Fix plugin update bug on broken plugin https://github.com/modmail-dev/Modmail/pull/3295/files * Update CHANGELOG.md * Fix rate limit issue on raw reaction add/remove events https://github.com/modmail-dev/Modmail/pull/3306/files * Remove Heroku Reference from Debug Command modmail-dev@d891ad1 * Update CHANGELOG.md * Remove unused files * Update pip * Bump aiohttp and use [speedups] * Update .dockerignore * Rename project * Add shell script for pm2 * Update docs * Update env example * Remove app.json * Update dependencies * Update readme * Add config to manage how long logs are stored (modmail-dev#3257) * Add presence intent into .env.example * Formating * Fix missing guild icon url * Revert gyazo image embedding * Fix overly permissive regex * Specify fork on metadata * Update changelog * Bump version --------- Co-authored-by: khakers <[email protected]> Co-authored-by: Taku <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Complete rewrite of how blocking is handled in modmail
The intention of this system is to unify blocking to one new mongodb collection and vastly simplify bot code required to manage it.
Provides migrations through the [p]migrate command for both legacy string and dict based user blocks. This should allow a seamless migration (though a manual one). Legacy string based block will create new entrys with a timestamp set to the migration time and with an invalid
blocking_user_id(0) since figuring these out would be both a lot of effort and generally useless due to discord username changes. The description of the migrated entry will be 'migrated from old format `{old block string}`'.I've added code to detect legacy blocks and log a warning telling the user how to migrate. It would also be possible to do an unattended automatic migration instead of manually through the command, but I was a bit concerned about unilaterally deciding to do so.