Feat/plugin install

.pre-commit-config.yaml

@@ -50,7 +50,7 @@ repos:
     - id: generate_requirements.txt
       name: Generate requirements.txt
       entry: python -m scripts.export_requirements --docs
-      files: '(pyproject.toml|poetry.lock|requirements.txt|scripts\/export\_requirements\.py|docs\/.requirements.txt)$'
+      files: '(pyproject.toml|poetry.lock|requirements.txt|constraints.txt|scripts\/export\_requirements\.py|docs\/.requirements.txt)$'
       language: python
       pass_filenames: false
       require_serial: true

Dockerfile

@@ -3,6 +3,9 @@ FROM python:3.9-slim
 # Set pip to have cleaner logs and no saved cache
 ENV PIP_NO_CACHE_DIR=false
 
+# Update pip
+RUN pip install -U pip
+
 # Create the working directory
 WORKDIR /modmail
 

docs/addons/README.md

@@ -0,0 +1,48 @@
+# Addons
+
+Addons are our built-in system to extend the features of the bot in an officially supported manner.
+
+Modmail, in its most basic form, is simple: relay messages to and from users to staff members.
+However, we acknowledge that its not a one-size-fits-all solution.
+Some communities need a few more features than others.
+That's where the addon system fills the void.
+
+The addon system currently supports only one kind of addon, plugins.
+This guide will help you set up a respository to create your own addons.
+Once its set up, please refer to the [plugin creation guide][making-plugins] for more details.
+
+!!!note
+    This guide is for those who want to **write** addons. If you are looking to use an addon, please view our guide [on installing them][installation].
+
+## Guides
+
+- [Installation]
+- [Repo Setup](#repo-setup)
+- [Creating Plugins][making-plugins]
+
+## Repo Setup
+
+In order to be able to install addons, a few things are required.
+Each addon type will have its own requirements in addition to the following.
+
+### Overall File Structure
+
+At the base of the addon system is the source. Sources have a folder structure like the following:
+
+```sh
+.
+├── Plugins/
+└── README.md
+```
+
+In this structure, this repository is holding addons of a plugin type. The structure of the Plugins folder itself is detailed in the [creating plugins guide][making-plugins].
+
+### Hosting
+
+All addons must be hosted on either github or gitlab as of now.
+
+!!!note
+    Addons currently do not automatically update, and need to be re-installed on each run. This will be fixed once the database client exists.
+
+[installation]: ./installation.md
+[making-plugins]: ./plugins.md

docs/addons/installation.md

@@ -0,0 +1,65 @@
+# Installation
+
+!!!note
+    If you are looking to write addons, check out our [writing addons][addon-guide] guide.
+
+## Plugins
+
+Plugins are discord.py extensions which expand the functionality of the bot beyond its core feature: relaying messages back and forth between staff and members.
+
+We've done our best to make this system easy to use for both novice and experienced developers--installing plugins should require no programming knowledge at all.
+
+By default, modmail will install plugins hosted on [github.com](https://github.com), but also supports installing from [gitlab](https://gitlab.com).
+
+This may look complex, but it supports a wide variety of options, as demonstrated below
+
+```fix
+?plugin install [git-host] <user>/<repo> <name> [@ref]
+?plugin install <link> <name> [@ref]
+```
+
+### Git-host style
+
+> `[git-host] <user>/<repo> <name> [@ref]`
+
+#### Git-host (Optional)
+
+Valid options are:
+
+- `github`
+- `gitlab`
+
+Default:
+
+- `github`
+
+#### User/Repo
+
+This is the user and the respository hosted on a valid git-host.
+
+In the link <https://github.com/discord-modmail/addons>, the user and repo are `discord-modmail/addons`.
+
+#### Name
+
+This is the addon name, it is not allowed to contain `@`.
+By default, this is the plugin folder name, unless it is defined in the plugin.toml file.
+A repository should provide a list of their plugins either in a plugin readme, or the full repository readme.
+
+#### Ref
+
+This is the git reference, leave blank to use the repository default.
+If you would like to use a specific commit, branch, or tag, then provide it preceeded by a `@`.
+For example, to use tagged version 1.2, `@v1.2` would install from that tag.
+
+### Link
+
+> `<link> <name> [@ref]`
+
+If the above githost format seems too complicated, its possible to just copy the url to the repo
+(ex. https://github.com/discord-modmail/addons) and use that for the link.
+
+The name of the plugin still must be provided, however.
+The @ref can also be provided, if installating a specific version is desired.
+
+
+[addon-guide]: ./README.md

docs/addons/plugins.md

@@ -0,0 +1,134 @@
+# Creating Plugins
+
+If you are looking to write a feature to extend the functionality of your modmail bot, plugins are *the*
+supported way to add additional code to modmail.
+
+In short, plugins are discord.py extensions which expand the functionality of the bot beyond its built-in duties.
+
+
+!!!Tip
+    This builds on the [addon structure documentation][addon-guide]. Please ensure you have a solid understanding of the basic repository structure beforehand.
+
+!!!note
+    This guide is **not** how to install plugins, please view our [installation guide][installation] for that.
+
+## File Structure Overview
+
+This details the structure of a plugin addon.
+
+```sh
+Plugins/
+├── react_to_contact
+│   ├── listener.py
+│   └── react_to_contact.py
+├── verify_contact
+│   └── verify_contact.py
+└── plugin.toml
+```
+
+Even though there are three `.py` files, this repository contains two plugins. Each top level folder in the Plugins folder contains one plugin.
+The number of py files in each plugins folder does not matter, there are still two plugins here.
+
+One plugin here is named `react_to_contact`, the other is `verify_contact`
+
+However, those are not user friendly names. It would be a lot easier for the end user to reference with `React to Contact`, and for the user interface to refer to it as such.
+
+To do so, a name can be provided in the plugin.toml file.
+
+## plugin.toml
+
+There are several variables which can be configured by providing a plugin.toml file.
+
+If you don't already know what toml is, [check out their docs](https://toml.io/)
+
+!!!tip
+    `plugin.toml` is supplemental to the list of folders. This means that all plugins in the repository are installable at any time. Providing a plugin.toml does not mean that any plugins *not* in the toml are not included anymore.
+
+    This has the advantage of being able to use `plugin.toml` to change the name of one plugin, without having to add all other plugins to the toml.
+
+
+### Options
+
+A full `plugin.toml` for the above repository may look like this:
+
+```toml
+[[plugins]]
+name = 'React to Contact'
+description = 'Provides a permanent message where the user can react to open a thread'
+directory = 'react_to_contact'
+
+[[plugins]]
+name = 'Verify Contact'
+description = 'Prevents the user from accidently opening a thread by asking if they are sure.'
+directory = 'verify_contact'
+```
+
+The name and directory are the only keys in use today,
+the description is not yet used.
+
+The `directory` key is required, if wanting to set any other settings for a plugin.
+
+!!!tip
+    `directory` is aliased to `folder`. Both keys are valid, but if the `directory` key exists it will be used and `folder` will be ignored.
+
+Name is optional, and defaults to the directory if not provided.
+
+!!!warning
+    Capitals matter. Both the `plugin.toml` file and `[[plugins]]` table ***must*** be lowercase.
+    This also goes for all keys and directory arguments--they must match the capitials of the existing directory.
+
+### Dependencies
+
+If the dependencies that the bot is installed with, it is possible to declare a dependency and it will be installed when installing the plugin.
+
+!!! Waring
+    For the most part, you won't need to use this. But if you absolutely must use an additional dependency which isn't part of the bot, put it in this array.
+
+This is an array of arguments which should be just like they are being passed to pip.
+
+```toml
+[[plugins]]
+directory = 'solar_system'
+dependencies = ['earthlib==0.2.2']
+```
+
+This will install earthlib 0.2.2.
+
+## Code
+
+Now that we have an understanding of where the plugin files go, and how to configure them, its time to write their code.
+
+### `PluginCog`
+
+All plugin cogs ***must*** inherit from `PluginCog`.
+
+If plugin cogs do not inherit from this class, they will fail to load.
+
+A majority of the needed modmail classes have been imported into helpers for your convinence.
+
+```python
+from modmail.addons import helpers
+
+# Cog
+helpers.PluginCog
+
+# Extension Metadata
+helpers.ExtMetadata
+
+### For Typehints
+# bot class
+helpers.ModmailBot
+
+# logger
+helpers.ModmailLogger
+```
+
+### `ExtMetadata`
+
+There is a system where extensions can declare load modes.
+
+There is a longer write up on it [here][ext_metadata].
+
+[addon-guide]: ./README.md
+[ext_metadata]: /contributing/creating_an_extension/#bot_mode-and-extmetadata
+[installation]: ./installation.md#plugins

docs/changelog.md

@@ -14,6 +14,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 - Threads system (#53)
+- Plugin installation and uninstall system (#69)
     - Messages can now be relayed between a user and a server.
     - NOTE: There is not a database yet, so none of these messages are stored.
 - Added Dispatcher system, although it is not hooked into important features like thread creation yet. (#71)
@@ -43,6 +44,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
     - Running the bot is still the same method, but it loads extensions and plugins now.
     - `bot.start()` can also be used if already in a running event loop. Keep in mind using it will require
         handling loop errors, as run() does this automatically.
+- Disabled some plugin management commands if PLUGIN_DEV mode is not set (#69)
 
 ### Internal
 

docs/contributing/creating_an_extension.md

@@ -0,0 +1,83 @@
+# Creating an Extension
+
+Welcome!
+
+Please note that extensions are cogs are different things. Extensions are files which add features to the bot,
+and a cog is way to group commands and features within a file.
+
+This is an addendum to the [discord.py guide](https://discordpy.readthedocs.io/en/master/ext/commands/extensions.html) on how to write extensions.
+This guide below details additional information which is not part of discord.py.
+
+**There is one major change from discord.py**:
+Cogs **must** inherit from `ModmailCog`.
+If this does not happen, the bot will let you know.
+
+ModmailCog is defined in `modmail/utils/cogs.py`.
+
+## BOT_MODE and `ExtMetadata`
+
+In general, an extension does not need to use the feature of an extension metadata.
+
+On every extension of the bot, an `EXT_METADATA` constant should exist, and it should be an instance of `ExtMetadata`.
+The `ExtMetadata` class is defined in `modmail/utils/cogs.py`, along with `BotModeEnum`.
+
+It should be sufficent to have an EXT_METADATA variable declared at the top of the file as an instance of ExtMetadata.
+
+```python
+from modmail.utils.cogs import ExtMetadata
+
+EXT_METADATA = ExtMetadata()
+```
+
+### `ExtMetadata`
+
+The purpose of ExtMetadata is to define metadata about an extension. Currently, it supports two items of metadata.
+
+- `load_if_mode`
+    - used to determine if this extension should be loaded at runtime.
+- `no_unload` (Not supported by plugins)
+    - prevents an extension from being unloaded by the `?ext unload` command. This is mainly used to keep the extension manager from unloading itself.
+
+`no_unload` is pretty self explanatory, pass either True or False and the extension will either be blocked from being unloaded, or allowed to unload.
+This only has an impact if the current bot mode is DEVELOP. Note that this does prevent the developer from *reloading* the extension.
+
+### `load_if_mode`
+
+`load_if_mode` currently has three modes, which each have their own uses.:
+
+- `PRODUCTION`
+    - The default mode, the bot is always in this mode.
+- `DEVELOP`
+    - Bot developer. Enables the extension management commands.
+- `PLUGIN_DEV`
+    - Plugin developer. Enables lower-level plugin commands.
+
+!!!tip
+    To enable these modes, set the corresponding environment variable to a truthy value. eg `DEVELOP=1` in your project `.env` file will enable the bot developer mode.
+
+To set an extension to only load on one cog, set the load_if_mode param when initialising a ExtMetadata object.
+
+```python
+from modmail.utils.cogs import BotModeEnum, ExtMetadata
+
+EXT_METADATA = ExtMetadata(load_if_mode=BotModeEnum.DEVELOP)
+```
+
+*This is not a complete extension and will not run if tested!*
+
+This `EXT_METADATA` variable above declares the extension will only run if a bot developer is running the bot.
+
+However, we may want to load our extension normally but have a command or two which only load in specific modes.
+
+### `BOT_MODE`
+
+The bot exposes a BOT_MODE variable which contains a bitmask of the current mode. This is created with the BotModeEnum.
+This allows code like this to determine if the bot mode is a specific mode.
+
+```python
+from modmail.utils.cogs import BOT_MODE, BotModeEnum
+
+is_plugin_dev_enabled = BOT_MODE & BotModeEnum.PLUGIN_DEV
+```
+
+This is used in the plugin_manager extension to determine if the lower-level commands which manage plugin extensions directly should be enabled.

mkdocs.yml

@@ -56,10 +56,18 @@ plugins:
 
 # Page tree
 nav:
-- Home: README.md
-- Contributing: contributing.md
-- Security: security.md
-- Changelog: changelog.md
+  - Home: README.md
+  - Changelog: changelog.md
+  - Security: security.md
+  - Addons:
+      - Overview: addons/README.md
+      - Installation: addons/installation.md
+      - Creating Plugins: addons/plugins.md
+  - Contributing:
+      - Guidelines: contributing.md
+      - Creating an Extension: contributing/creating_an_extension.md
+
+
 
 # Extensions
 markdown_extensions: