Skip to content

Commit b969a54

Browse files
immanuelfodorimmanuelfodor
committed
Add readme file
1 parent d91ce7f commit b969a54

File tree

1 file changed

+149
-0
lines changed

1 file changed

+149
-0
lines changed

‎README.md‎

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
# End-to-end encrypted (E2EE) Matrix Webhook Gateway <!-- omit in toc -->
2+
3+
A microservice that forwards form data or JSON objects received in an HTTP POST request to an end-to-end-encrypted (E2EE) Matrix room. There is no schema restriction on the posted data, so you can throw at it _anyting_! Supports multiple rooms and sender aliases with different associated webhook URLs, e.g, one for Grafana, another for `curl`, and so on. Can convert the received payload to JSON or YAML for better message readability, and can apply Markdown formatting to messages. Easy installation with `docker-compose`.
4+
5+
## Table of contents <!-- omit in toc -->
6+
7+
- [Usage](#usage)
8+
- [Configuration](#configuration)
9+
- [Available customizations of posted messages](#available-customizations-of-posted-messages)
10+
- [Matrix connection parameters](#matrix-connection-parameters)
11+
- [The notification channels](#the-notification-channels)
12+
- [Dependencies](#dependencies)
13+
- [Development](#development)
14+
- [Troubleshooting](#troubleshooting)
15+
- [Improvement ideas](#improvement-ideas)
16+
- [Disclaimer](#disclaimer)
17+
- [Contact](#contact)
18+
19+
## Usage
20+
21+
- Create a new Matrix account on your homeserver for receiving webhooks
22+
- Add this account to a new E2EE room with yourself then login with the new account and accept the invite
23+
- Clone this repo, create a copy of the provided `.env.example` file as `.env`, and fill in all the required info
24+
- Start up the gateway. It logs in, sends a greeting message, and listens to webhook events
25+
- Send data with `curl` or any HTTP POST capable client to the gateway
26+
- Enjoy the messages in your chat!
27+
28+
```bash
29+
cp .env{.example,}
30+
# edit the .env with your preferred editor
31+
32+
mkdir -p store
33+
chown 1000:1000 store
34+
sudo docker-compose up -d --build
35+
36+
# post form data
37+
curl -d 'hello="world"' http://localhost:8000/post/YOURSECRETTOKEN
38+
39+
# post a JSON
40+
curl -H 'Content-Type: application/json' -d '{"hello":"world"}' http://localhost:8000/post/YOURSECRETTOKEN
41+
42+
# post a text file by converting it to a valid JSON array
43+
cat /etc/resolv.conf | jq -R -s -c 'split("\n")' | curl -H 'Content-Type: application/json' -d @- http://localhost:8000/post/YOURSECRETTOKEN
44+
45+
# make posts cleaner by adding all parameters to a curl config file
46+
printf '# @see: https://ec.haxx.se/cmdline-configfile.html
47+
url = "http://localhost:8000/post/YOURSECRETTOKEN"
48+
header = "Content-Type: application/json"
49+
output = /dev/null
50+
silent
51+
' > ~/.matrix.curlrc
52+
/any/command | curl -K ~/.matrix.curlrc -d @-
53+
```
54+
55+
## Configuration
56+
57+
The gateway tries to join all of the specified rooms in the `.env` file on start. However, you must make sure to invite the webhook user and accept the invite on behalf of them from any client before you start up the gateway!
58+
59+
### Available customizations of posted messages
60+
61+
- Message formatting via `MESSAGE_FORMAT`: `raw` | `json` | `yaml` (default)
62+
- Markdown formatting turned on or off via `USE_MARKDOWN`: `True` | `False` (default)
63+
- ASCII (e.g., `\u1234`) or Unicode characters (e.g., `ű`) in JSON or YAML content via `ALLOW_UNICODE`: `True` (default) | `False`
64+
65+
### Matrix connection parameters
66+
67+
- The URL of the Matrix server via `MATRIX_SERVER`: a string like `https://matrix.example.org`
68+
- SSL cert checking turned on or off via `MATRIX_SSLVERIFY`: `True` (default) | `False`
69+
- The webhook user ID via `MATRIX_USERID`: a string like `@myhook:matrix.example.org`
70+
- The webhook user password via `MATRIX_PASSWORD`: a string like `mypass+*!word`
71+
- Put the string in quotes if your password contains likely sell-unsafe characters
72+
- A device name that the webhook user will use via `MATRIX_DEVICE`: a string like `docker`
73+
- The room where the webhook user will send its greeting upon (re)start via `MATRIX_ADMIN_ROOM`: a string like `!privatechatwiththebotuser:matrix.example.org`
74+
- You must use the room ID (with `!`) at all times even if the room has an alias like `#alias:matrix.example.org`!
75+
- You can invite the webhook user to a private chat with yourself to get a different room than the ones you're using for notification, it's up to you. Later, we can add some basic commands for the webhook service as it can read messages in joined rooms, so I made the admin room a private chat with the bot to be future-proof.
76+
77+
### The notification channels
78+
79+
- The list of `token,roomid,name` triplets separated by spaces via `KNOWN_TOKENS`: a string like
80+
`YOURSECRETTOKEN,!myroomid:matrix.example.org,Curl anOTheRToKen99,!myroomid2:matrix.example.org,Grafana`
81+
- Put the string in quotes if you have more than one triplet ie. you have at least one space on the line
82+
- The `token` and `name` parts should match the following regexp: `[a-zA-Z0-9]+`
83+
- You must use the room ID (with `!`) at all times even if the room has an alias like `#alias:matrix.example.org`!
84+
- You can use different room IDs for different notifications, and these can be different from the admin room as well. But you can use the same room for everything, it depends on your use case.
85+
86+
### "Hidden" parameters that need no change most of the time <!-- omit in toc -->
87+
88+
- The Python log level via `PYTHON_LOG_LEVEL`: `debug` | `info` (default) | `warn` | `error` | `critical`
89+
- The store path of the saved login session via `LOGIN_STORE_PATH`: a path-string without trailing slash like `/config` (default)
90+
- If you want to make login sessions persist to avoid device IDs stacking up on the webhook user, you can put this path on a docker volume with read-write permissions to the `1000:1000` user:group.
91+
92+
## Dependencies
93+
94+
- [Docker](https://www.docker.com/)
95+
- [Docker Compose](https://github.com/docker/compose)
96+
97+
Installation on Manjaro Linux:
98+
99+
```bash
100+
sudo pacman -S docker docker-compose
101+
102+
sudo systemctl enable --now docker
103+
```
104+
105+
Of course, you'll also need a [Matrix server](https://matrix.org/discover/) up and running with at least one E2EE room and two users joined in the same room (the webhook user and probably _you_). Explaining setting these up is way beyond the scope of this document, so please see the online docs for proper instructions, or use a hosted Matrix server.
106+
107+
## Development
108+
109+
Install global dependencies, install dev dependencies, create a new virtual environment, install package dependencies, then start the project:
110+
111+
```bash
112+
# Manjaro
113+
sudo pacman -S libolm
114+
sudo pip install virtualenv
115+
116+
virtualenv venv
117+
118+
# Fish
119+
source venv/bin/activate.fish
120+
121+
pip install -r requirements.txt
122+
123+
./docker-entrypoint.sh
124+
```
125+
126+
## Troubleshooting
127+
128+
- You might need to turn off SSL cert verification (with `False` in the environment file) if your certs are self-signed or you experience any problem with the cert verification even if you have a valid cert.
129+
- Depending on the content of your messages, you might need to experiment with the available formatting combinations that suits your use case and eyes. We are different, I like plain YAML best with Unicode characters.
130+
131+
## Improvement ideas
132+
133+
- Execute predefined commands from the admin room for a set of predefined admin users. Maubot seems to be more flexible in this case but we could implement simple controls for the webhok gateway itself.
134+
135+
## Disclaimer
136+
137+
**This is an experimental project. I do not take responsibility for anything regarding the use or misuse of the contents of this repository.**
138+
139+
- Tested with Grafana webhooks and Synapse as a Matrix server, but in theory, it should work with any source capable of sending HTTP POST requests with valid form data or JSON objects (e.g., `curl`).
140+
- Tested in rooms only with E2EE enabled as the main goal of this project is to receive arbitrary data in encrypted rooms.
141+
- JSON and YAML formatting is indented by 2 spaces, no further processing is being made. The padded object's size is limited by your server's maximum message size.
142+
- If you expose this service to the net, malicious actors could only send spam notifications if they knew any of your long and random tokens, as the token acts as auth. Otherwise, the messages are not routed to a Matrix room, so you're safe. However, if you host your message source and this gateway on the same network, you can use local hostnames, DNS or IP addresses to set the webhook up in the source. This way the message gateway won't be accessible from the outside, and your message data won't leave the internal network without encryption.
143+
- If you use RocketChat, there is a similar project for that service here: [immanuelfodor/rocketchat-push-gateway](https://github.com/immanuelfodor/rocketchat-push-gateway)
144+
- If you use XMPP, there is a similar project for that service here: [immanuelfodor/xmpp-muc-message-gateway](https://github.com/immanuelfodor/xmpp-muc-message-gateway)
145+
146+
## Contact
147+
148+
Immánuel Fodor
149+
[fodor.it](https://fodor.it/matrixmsgwit) | [Linkedin](https://fodor.it/matrixmsgwin)

0 commit comments

Comments
 (0)