Skip to content

Commit

Permalink
fix markdown linter errors
Browse files Browse the repository at this point in the history
  • Loading branch information
@chrisrude
chrisrude committed May 4, 2023
1 parent 8131eae commit b2dffd6
Show file tree
Hide file tree
Showing 2 changed files with 86 additions and 64 deletions.
108 changes: 62 additions & 46 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,34 +1,41 @@
# OobaDiscordBot
A Discord bot which talks to a Large Language Models AI (like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA), running on [oobabooga's text-generation-webui](https://github.com/oobabooga/text-generation-webui).
# `oobabot`

**`oobabot`** is a Discord bot which talks to a Large Language Model AIs (like LLaMA, llama.cpp, etc...), running on [oobabooga's text-generation-webui](https://github.com/oobabooga/text-generation-webui).

[![python lint and test with poetry](https://github.com/chrisrude/oobabot/actions/workflows/python-package.yml/badge.svg)](https://github.com/chrisrude/oobabot/actions/workflows/python-package.yml)

## Motivation

![oobabot in action!](./docs/oobabot.png "discord action shot")

# Motivation
Text-generative UIs are cool to run at home, and Discord is fun to mess with your friends. Why not combine the two and have something awesome!

Real motivation: I wanted a chatbot in my discord that would act like my cat. A "catbot" if you will.

# Features
- user-supplied persona! -- you supply the persona on how would like the bot to behave
- multiple converations at once -- can track multiple conversational threads, and reply to each in a contextually appropriate way
- watchwords -- can monitor all channels in a server for one or more wakewords or @-mentions
- private conversations -- can chat with you 1:1 in a DM
- good discord hygiene -- splits messages into independent sentences, pings the author in the first one
- low-latency -- streams the reply live, sentence by sentence. Provides lower latency, especially on longer responses.
- stats -- track token generation speed, latency, failures and usage
- easy networking -- connects to discord from your machine using websockets, so no need to expose a server to the internet
## Features

| **`oobabot`** | how that's awesome |
|---------------|------------------|
| **user-supplied persona** | you supply the persona on how would like the bot to behave |
| **multiple converations** | can track multiple conversational threads, and reply to each in a contextually appropriate way |
| **watchwords** | can monitor all channels in a server for one or more wakewords or @-mentions |
| **private conversations** | can chat with you 1:1 in a DM |
| **good Discord hygiene** | splits messages into independent sentences, pings the author in the first one |
| **low-latency** | streams the reply live, sentence by sentence. Provides lower latency, especially on longer responses. |
| **stats** | track token generation speed, latency, failures and usage |
| **easy networking** | connects to discord from your machine using websockets, so no need to expose a server to the internet |

## Getting Started with **`oobabot`**

# Getting Started with Oobabot
### See the [Installation Guide](./docs/INSTALL.md) for step-by-step instructions

### See the [Installation Guide](./docs/INSTALL.md) for step-by-step instructions.
## Installation tl;dr

## Installation tl;dr:
1. Install [oobabooga's text-generation-webui](https://github.com/oobabooga/text-generation-webui) and enable its API module
1. Create [a Discord bot account](https://discordpy.readthedocs.io/en/stable/discord.html), invite it to your server, and note its authentication token.
1. [Install oobabot (see INSTALL.md)](./docs/INSTALL.md)
```
1. [Install **`oobabot`** (see INSTALL.md)](./docs/INSTALL.md)

```bash
~: pip install oobabot

~: export DISCORD_TOKEN = __your_bots_discord_token__
Expand All @@ -38,7 +45,7 @@ Real motivation: I wanted a chatbot in my discord that would act like my cat. A

You should now be able to run oobabot from whenever pip installed it.

```
```none
usage: oobabot [-h] [--base-url BASE_URL] [--ai-name AI_NAME] [--wakewords [WAKEWORDS ...]]
[--persona PERSONA] [--local-repl] [--log-all-the-things LOG_ALL_THE_THINGS]
Expand Down Expand Up @@ -67,51 +74,60 @@ Also, to authenticate to Discord, you must set the environment variable: DISCORD
discord token>
```

---
## Required settings:
- **`DISCORD_TOKEN`** environment variable
## Required settings

- **`DISCORD_TOKEN`** environment variable

Set your shell environment's **`DISCORD_TOKEN`** to token Discord provided when you set up the bot account. It should be something like a 72-character-long random-looking string.

#### Bash shell
export DISCORD_TOKEN=___YOUR_TOKEN___
**bash example**

``` bash
export DISCORD_TOKEN=___YOUR_TOKEN___
```

#### Fish shell
set -Ux DISCORD_TOKEN ___YOUR_TOKEN___
**fish example**

- **`--base-url`**
``` fish
set -Ux DISCORD_TOKEN ___YOUR_TOKEN___
```

- **`--base-url`**

The base URL of oobabooga's streaming web API. This is
required if the oobabooga machine is different than where you're running oobabot.
required if the oobabooga machine is different than where you're running **`oobabot`**.

By default, this will be port 5005 (even though the HTML UI runs on a different port). The protocol should typically be ws://.

All together, this should look something like:

`--base-url ws://localhost:5005`
```bash
--base-url ws://localhost:5005
```

This is also the default value, but any other setting should follow the same form.

## Optional settings:
- **`--ai-name`**
## Optional settings

- **`--ai-name`**

the name the AI will be instructed to call itself. Note that this technically doesn't need to be the same as the bot in your discord, but it would likely make sense to your users if they are at least similar.
- **`--wakewords`**
- **`--wakewords`**
one or more words that the bot will look for. It will reply to any message which contains one of these words, in any channel.
- **`--local-repl`**
- **`--local-repl`**
instead of connecting to discord, just start up a local REPL and send these prompts directly to the oobabooga server. Useful if you want to test if that part is working in isolation. Note that in this mode you will be sending the oobabooga server raw input, and so the persona or AI name settings will be ignored.
## Persona: the fun setting!
## Persona: the fun setting
- **`--persona`**
is a short few sentences describing the role your bot should act as. For instance, this is the setting I'm using for my cat-bot, whose name is "Rosie".

```
```console
Here is some background information about Rosie:
- You are Rosie
- You are a cat
Expand All @@ -123,29 +139,29 @@ Here is some background information about Rosie:
- Your personality is both witty and profane
- The people in this chat room are your friends
```

Persona may be set from the command line with the **`--persona`** argument.

Alternatively, it can be set through the environment variable **`OOBABOT_PERSONA`**.

## Then, run it!
## Then, run it

You should see something like this if everything worked:

![oobabot running!](./docs/oobabot-cli.png "textually interesting image")

---

# Interacting with oobabot

By default, oobabot will listen for three types of messages in the servers it's connected to:
## Interacting with **`oobabot`**

1. any message in which oobabot's account is @-mentioned
2. any direct message
3. any message containing a provided wakeword (see Optional Settings)
By default, **`oobabot`** will listen for three types of messages in the servers it's connected to:
1. any message in which **`oobabot`**'s account is @-mentioned
1. any direct message
1. any message containing a provided wakeword (see Optional Settings)

# Known Issues
- ooba's text generation can errs with OOM when more than one request comes in at once.
- sometimes the bot wants to continue conversations on behalf of other members of the chatroom. I have some hacks in place to notice and truncate this behavior, but it can lead to terse responses on occasion.
- found one not listed here? [Create an issue](https://github.com/chrisrude/oobabot/issues) on github!
## Known Issues

- ooba's text generation can errs with OOM when more than one request comes in at once.
- sometimes the bot wants to continue conversations on behalf of other members of the chatroom. I have some hacks in place to notice and truncate this behavior, but it can lead to terse responses on occasion.
- found one not listed here? [Create an issue](https://github.com/chrisrude/oobabot/issues) on github!
42 changes: 24 additions & 18 deletions docs/INSTALL.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,38 +5,44 @@ To get this working, you'll need three things up and talking to each other:

## Fancy-Pants Architecture Diagram

Here's what you'll be setting up.
Here's what you'll be setting up:

[ text-generation-webui ] <-- [ oobabot ] --> [ discord bot account ]

(--> means: connects to via websockets)
```none
[ text-generation-webui ] <-- [ oobabot ] --> [ Discord's servers ]
[ your pc ] [ your pc ] [ the internet ]
```

`[a]-->[b]` means: `a` connects to `b` via websockets

## 1. Install `text-generation-webui`

- get the text-generation-webui running on your box. I used a few guides to do this:
- https://www.reddit.com/r/LocalLLaMA/comments/11o6o3f/how_to_install_llama_8bit_and_4bit/
- [`u/Technical_Leather949`'s *How to install Llama 8bit and 4bit*](https://www.reddit.com/r/LocalLLaMA/comments/11o6o3f/how_to_install_llama_8bit_and_4bit/) on reddit
- the instructions on [oobabooga's text-generation-webui github](https://github.com/oobabooga/text-generation-webui)
- download a model to run. I suspect the most interesting model will change frequenty, but as of May 3 20213 I am currently using https://huggingface.co/chavinlo/gpt4-x-alpaca. This runs quite well a GPU with 10GB of RAM.
- enable the "API" plugin. You can do this with the '--api' command-line option, or by enabling the "api" plugin on the "interface mode" tab of the web UI.
- make note of the URL to the web UI, you'll need this in a later step (either http or https should work, your choice)

## 2. Create a Discord bot account for oobabot
- download a model to run.

I suspect the most interesting model will change frequenty, but as of May 3 20213 I am currently using [gpt4-x-alpaca, found on HuggingFace](https://huggingface.co/chavinlo/gpt4-x-alpaca). This runs quite well a GPU with 10GB of RAM.
- enable the "API" plugin. You can do this with the `--api` command-line option, or by enabling the "api" plugin onfrom the "interface mode" tab of the web UI.
- make note of the URL to the web UI, you'll need this in a later step (either ws:// or wss:// should work, your choice)

## 2. Create a Discord bot account for **`oobabot`**

You can follow the steps in [discord.py's documentation](https://discordpy.readthedocs.io/en/stable/discord.html), [Discord's own documentation](https://discord.com/developers/docs/getting-started) or follow any number of online guides.
It boils down to first, creating a bot account:

- log into the discord web interface (the native apps don't expose these settings)
- go to the [application page](https://discord.com/developers/applications)
- create a new application.
- Choose a name that matches what you want the bot to be called. This will be visible to users.
- Public bot / private bot doesn't matter
- generate a token for the bot. MAKE NOTE OF THIS FOR LATER.
- enable the bot's intents as follows:
- PRESENCE INTENT: OFF
- SERVER MEMBERS INTENT: ON
- MESSAGE CONTENT INTENT: ON
- `PRESENCE INTENT: OFF`
- `SERVER MEMBERS INTENT: ON`
- `MESSAGE CONTENT INTENT: ON`

## 3. Invite oobabot to Discord servers
## 3. Invite **`oobabot`** to Discord servers

- go to the [application page](https://discord.com/developers/applications)
- click on your bot’s page, then the "OAuth2" tab
Expand All @@ -51,7 +57,7 @@ It boils down to first, creating a bot account:
- ✅ read mention history
- *disable everything else*
- **Voice Permissions**
- *disable everything*
- *disable everything*
- generate the URL
- give the URL to an admin on the Discord servers of interest, and have them accept the various warnings that will show up

Expand All @@ -60,13 +66,13 @@ It boils down to first, creating a bot account:
You can install oobabot on any machine that can reach the oobabooga's text-generation-webui URL you noted above. By default it will assume it's the same machine, but you can also run it anywhere else if that's more convenient.

Using python3.8, 3.9 or 3.10, install the `oobabot` package from PiPy, using your favorite package manager. E.g.
```

```bash
pip install oobabot
```

## 5. Configure oobabot amd have fun!
## 5. Configure oobabot and have fun

Please refer to the [main README.md](../README.md) for setup instructions.

Oobabot can be a lot of fun for a discord to talk to, especially with a creative personality. Be creative and enjoy it!

**`oobabot`** can be a lot of fun for a discord to talk to, especially with a creative personality. Be creative and enjoy it!

0 comments on commit b2dffd6

Please sign in to comment.