Skip to content

Commit a2a475b

Browse files
HenryHengZJHenryHengZJ
authored
Feature/Add bullmq redis for message queue processing (#3568)
* add bullmq redis for message queue processing * Update pnpm-lock.yaml * update queue manager * remove singleton patterns, add redis to cache pool * add bull board ui * update rate limit handler * update redis configuration * Merge add rate limit redis prefix * update rate limit queue events * update preview loader to queue * refractor namings to constants * update env variable for queue * update worker shutdown gracefully
1 parent 14adb93 commit a2a475b

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

59 files changed

+38739
-36766
lines changed

Diff for: ‎CONTRIBUTING.md

+39-40
Large diffs are not rendered by default.

Diff for: ‎docker/.env.example

+18-3
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,6 @@ BLOB_STORAGE_PATH=/root/.flowise/storage
3232
# FLOWISE_SECRETKEY_OVERWRITE=myencryptionkey
3333
# FLOWISE_FILE_SIZE_LIMIT=50mb
3434

35-
# DISABLE_CHATFLOW_REUSE=true
36-
3735
# DEBUG=true
3836
# LOG_LEVEL=info (error | warn | info | verbose | debug)
3937
# TOOL_FUNCTION_BUILTIN_DEP=crypto,fs
@@ -79,4 +77,21 @@ BLOB_STORAGE_PATH=/root/.flowise/storage
7977
# see https://www.npmjs.com/package/global-agent for more details
8078
# GLOBAL_AGENT_HTTP_PROXY=CorporateHttpProxyUrl
8179
# GLOBAL_AGENT_HTTPS_PROXY=CorporateHttpsProxyUrl
82-
# GLOBAL_AGENT_NO_PROXY=ExceptionHostsToBypassProxyIfNeeded
80+
# GLOBAL_AGENT_NO_PROXY=ExceptionHostsToBypassProxyIfNeeded
81+
82+
######################
83+
# QUEUE CONFIGURATION
84+
#######################
85+
# MODE=queue #(queue | main)
86+
# QUEUE_NAME=flowise-queue
87+
# QUEUE_REDIS_EVENT_STREAM_MAX_LEN=100000
88+
# WORKER_CONCURRENCY=100000
89+
# REDIS_URL=
90+
# REDIS_HOST=localhost
91+
# REDIS_PORT=6379
92+
# REDIS_USERNAME=
93+
# REDIS_PASSWORD=
94+
# REDIS_TLS=
95+
# REDIS_CERT=
96+
# REDIS_KEY=
97+
# REDIS_CA=

Diff for: ‎docker/docker-compose.yml

+13
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,19 @@ services:
3434
- GLOBAL_AGENT_HTTPS_PROXY=${GLOBAL_AGENT_HTTPS_PROXY}
3535
- GLOBAL_AGENT_NO_PROXY=${GLOBAL_AGENT_NO_PROXY}
3636
- DISABLED_NODES=${DISABLED_NODES}
37+
- MODE=${MODE}
38+
- WORKER_CONCURRENCY=${WORKER_CONCURRENCY}
39+
- QUEUE_NAME=${QUEUE_NAME}
40+
- QUEUE_REDIS_EVENT_STREAM_MAX_LEN=${QUEUE_REDIS_EVENT_STREAM_MAX_LEN}
41+
- REDIS_URL=${REDIS_URL}
42+
- REDIS_HOST=${REDIS_HOST}
43+
- REDIS_PORT=${REDIS_PORT}
44+
- REDIS_PASSWORD=${REDIS_PASSWORD}
45+
- REDIS_USERNAME=${REDIS_USERNAME}
46+
- REDIS_TLS=${REDIS_TLS}
47+
- REDIS_CERT=${REDIS_CERT}
48+
- REDIS_KEY=${REDIS_KEY}
49+
- REDIS_CA=${REDIS_CA}
3750
ports:
3851
- '${PORT}:${PORT}'
3952
volumes:

Diff for: ‎docker/worker/README.md

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Flowise Worker
2+
3+
By utilizing worker instances when operating in queue mode, Flowise can be scaled horizontally by adding more workers to handle increased workloads or scaled down by removing workers when demand decreases.
4+
5+
Here’s an overview of the process:
6+
7+
1. The primary Flowise instance sends an execution ID to a message broker, Redis, which maintains a queue of pending executions, allowing the next available worker to process them.
8+
2. A worker from the pool retrieves a message from Redis.
9+
The worker starts execute the actual job.
10+
3. Once the execution is completed, the worker alerts the main instance that the execution is finished.
11+
12+
# How to use
13+
14+
## Setting up Main Server:
15+
16+
1. Follow [setup guide](https://github.com/FlowiseAI/Flowise/blob/main/docker/README.md)
17+
2. In the `.env.example`, setup all the necessary env variables for `QUEUE CONFIGURATION`
18+
19+
## Setting up Worker:
20+
21+
1. Copy paste the same `.env` file used to setup main server. Change the `PORT` to other available port numbers. Ex: 5566
22+
2. `docker compose up -d`
23+
3. Open [http://localhost:5566](http://localhost:5566)
24+
4. You can bring the worker container down by `docker compose stop`

Diff for: ‎docker/worker/docker-compose.yml

+54
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
version: '3.1'
2+
3+
services:
4+
flowise:
5+
image: flowiseai/flowise
6+
restart: always
7+
environment:
8+
- PORT=${PORT}
9+
- CORS_ORIGINS=${CORS_ORIGINS}
10+
- IFRAME_ORIGINS=${IFRAME_ORIGINS}
11+
- FLOWISE_USERNAME=${FLOWISE_USERNAME}
12+
- FLOWISE_PASSWORD=${FLOWISE_PASSWORD}
13+
- FLOWISE_FILE_SIZE_LIMIT=${FLOWISE_FILE_SIZE_LIMIT}
14+
- DEBUG=${DEBUG}
15+
- DATABASE_PATH=${DATABASE_PATH}
16+
- DATABASE_TYPE=${DATABASE_TYPE}
17+
- DATABASE_PORT=${DATABASE_PORT}
18+
- DATABASE_HOST=${DATABASE_HOST}
19+
- DATABASE_NAME=${DATABASE_NAME}
20+
- DATABASE_USER=${DATABASE_USER}
21+
- DATABASE_PASSWORD=${DATABASE_PASSWORD}
22+
- DATABASE_SSL=${DATABASE_SSL}
23+
- DATABASE_SSL_KEY_BASE64=${DATABASE_SSL_KEY_BASE64}
24+
- APIKEY_STORAGE_TYPE=${APIKEY_STORAGE_TYPE}
25+
- APIKEY_PATH=${APIKEY_PATH}
26+
- SECRETKEY_PATH=${SECRETKEY_PATH}
27+
- FLOWISE_SECRETKEY_OVERWRITE=${FLOWISE_SECRETKEY_OVERWRITE}
28+
- LOG_LEVEL=${LOG_LEVEL}
29+
- LOG_PATH=${LOG_PATH}
30+
- BLOB_STORAGE_PATH=${BLOB_STORAGE_PATH}
31+
- DISABLE_FLOWISE_TELEMETRY=${DISABLE_FLOWISE_TELEMETRY}
32+
- MODEL_LIST_CONFIG_JSON=${MODEL_LIST_CONFIG_JSON}
33+
- GLOBAL_AGENT_HTTP_PROXY=${GLOBAL_AGENT_HTTP_PROXY}
34+
- GLOBAL_AGENT_HTTPS_PROXY=${GLOBAL_AGENT_HTTPS_PROXY}
35+
- GLOBAL_AGENT_NO_PROXY=${GLOBAL_AGENT_NO_PROXY}
36+
- DISABLED_NODES=${DISABLED_NODES}
37+
- MODE=${MODE}
38+
- WORKER_CONCURRENCY=${WORKER_CONCURRENCY}
39+
- QUEUE_NAME=${QUEUE_NAME}
40+
- QUEUE_REDIS_EVENT_STREAM_MAX_LEN=${QUEUE_REDIS_EVENT_STREAM_MAX_LEN}
41+
- REDIS_URL=${REDIS_URL}
42+
- REDIS_HOST=${REDIS_HOST}
43+
- REDIS_PORT=${REDIS_PORT}
44+
- REDIS_PASSWORD=${REDIS_PASSWORD}
45+
- REDIS_USERNAME=${REDIS_USERNAME}
46+
- REDIS_TLS=${REDIS_TLS}
47+
- REDIS_CERT=${REDIS_CERT}
48+
- REDIS_KEY=${REDIS_KEY}
49+
- REDIS_CA=${REDIS_CA}
50+
ports:
51+
- '${PORT}:${PORT}'
52+
volumes:
53+
- ~/.flowise:/root/.flowise
54+
entrypoint: /bin/sh -c "sleep 3; flowise worker"

Diff for: ‎i18n/CONTRIBUTING-ZH.md

+34-35
Original file line numberDiff line numberDiff line change
@@ -118,41 +118,40 @@ Flowise 在一个单一的单体存储库中有 3 个不同的模块。
118118

119119
Flowise 支持不同的环境变量来配置您的实例。您可以在 `packages/server` 文件夹中的 `.env` 文件中指定以下变量。阅读[更多信息](https://docs.flowiseai.com/environment-variables)
120120

121-
| 变量名 | 描述 | 类型 | 默认值 |
122-
| ---------------------------- | -------------------------------------------------------------------- | ----------------------------------------------- | ----------------------------------- |
123-
| PORT | Flowise 运行的 HTTP 端口 | 数字 | 3000 |
124-
| FLOWISE_USERNAME | 登录用户名 | 字符串 | |
125-
| FLOWISE_PASSWORD | 登录密码 | 字符串 | |
126-
| FLOWISE_FILE_SIZE_LIMIT | 上传文件大小限制 | 字符串 | 50mb |
127-
| DISABLE_CHATFLOW_REUSE | 强制为每次调用创建一个新的 ChatFlow，而不是重用缓存中的现有 ChatFlow | 布尔值 | |
128-
| DEBUG | 打印组件的日志 | 布尔值 | |
129-
| LOG_PATH | 存储日志文件的位置 | 字符串 | `your-path/Flowise/logs` |
130-
| LOG_LEVEL | 日志的不同级别 | 枚举字符串: `error`, `info`, `verbose`, `debug` | `info` |
131-
| APIKEY_STORAGE_TYPE | 存储 API 密钥的存储类型 | 枚举字符串: `json`, `db` | `json` |
132-
| APIKEY_PATH | 存储 API 密钥的位置, 当`APIKEY_STORAGE_TYPE`是`json` | 字符串 | `your-path/Flowise/packages/server` |
133-
| TOOL_FUNCTION_BUILTIN_DEP | 用于工具函数的 NodeJS 内置模块 | 字符串 | |
134-
| TOOL_FUNCTION_EXTERNAL_DEP | 用于工具函数的外部模块 | 字符串 | |
135-
| DATABASE_TYPE | 存储 flowise 数据的数据库类型 | 枚举字符串: `sqlite`, `mysql`, `postgres` | `sqlite` |
136-
| DATABASE_PATH | 数据库保存的位置（当 DATABASE_TYPE 是 sqlite 时） | 字符串 | `your-home-dir/.flowise` |
137-
| DATABASE_HOST | 主机 URL 或 IP 地址（当 DATABASE_TYPE 不是 sqlite 时） | 字符串 | |
138-
| DATABASE_PORT | 数据库端口（当 DATABASE_TYPE 不是 sqlite 时） | 字符串 | |
139-
| DATABASE_USERNAME | 数据库用户名（当 DATABASE_TYPE 不是 sqlite 时） | 字符串 | |
140-
| DATABASE_PASSWORD | 数据库密码（当 DATABASE_TYPE 不是 sqlite 时） | 字符串 | |
141-
| DATABASE_NAME | 数据库名称（当 DATABASE_TYPE 不是 sqlite 时） | 字符串 | |
142-
| SECRETKEY_PATH | 保存加密密钥（用于加密/解密凭据）的位置 | 字符串 | `your-path/Flowise/packages/server` |
143-
| FLOWISE_SECRETKEY_OVERWRITE | 加密密钥用于替代存储在 SECRETKEY_PATH 中的密钥 | 字符串 |
144-
| DISABLE_FLOWISE_TELEMETRY | 关闭遥测 | 字符串 |
145-
| MODEL_LIST_CONFIG_JSON | 加载模型的位置 | 字符 | `/your_model_list_config_file_path` |
146-
| STORAGE_TYPE | 上传文件的存储类型 | 枚举字符串: `local`, `s3` | `local` |
147-
| BLOB_STORAGE_PATH | 上传文件存储的本地文件夹路径, 当`STORAGE_TYPE`是`local` | 字符串 | `your-home-dir/.flowise/storage` |
148-
| S3_STORAGE_BUCKET_NAME | S3 存储文件夹路径, 当`STORAGE_TYPE`是`s3` | 字符串 | |
149-
| S3_STORAGE_ACCESS_KEY_ID | AWS 访问密钥 (Access Key) | 字符串 | |
150-
| S3_STORAGE_SECRET_ACCESS_KEY | AWS 密钥 (Secret Key) | 字符串 | |
151-
| S3_STORAGE_REGION | S3 存储地区 | 字符串 | |
152-
| S3_ENDPOINT_URL | S3 端点 URL | 字符串 | |
153-
| S3_FORCE_PATH_STYLE | 将其设置为 true 以强制请求使用路径样式寻址 | 布尔值 | false |
154-
| SHOW_COMMUNITY_NODES | 显示由社区创建的节点 | 布尔值 | |
155-
| DISABLED_NODES | 从界面中隐藏节点（以逗号分隔的节点名称列表） | 字符串 | |
121+
| 变量名 | 描述 | 类型 | 默认值 |
122+
| ---------------------------- | ------------------------------------------------------- | ----------------------------------------------- | ----------------------------------- | --- |
123+
| PORT | Flowise 运行的 HTTP 端口 | 数字 | 3000 |
124+
| FLOWISE_USERNAME | 登录用户名 | 字符串 | |
125+
| FLOWISE_PASSWORD | 登录密码 | 字符串 | |
126+
| FLOWISE_FILE_SIZE_LIMIT | 上传文件大小限制 | 字符串 | 50mb | |
127+
| DEBUG | 打印组件的日志 | 布尔值 | |
128+
| LOG_PATH | 存储日志文件的位置 | 字符串 | `your-path/Flowise/logs` |
129+
| LOG_LEVEL | 日志的不同级别 | 枚举字符串: `error`, `info`, `verbose`, `debug` | `info` |
130+
| APIKEY_STORAGE_TYPE | 存储 API 密钥的存储类型 | 枚举字符串: `json`, `db` | `json` |
131+
| APIKEY_PATH | 存储 API 密钥的位置, 当`APIKEY_STORAGE_TYPE`是`json` | 字符串 | `your-path/Flowise/packages/server` |
132+
| TOOL_FUNCTION_BUILTIN_DEP | 用于工具函数的 NodeJS 内置模块 | 字符串 | |
133+
| TOOL_FUNCTION_EXTERNAL_DEP | 用于工具函数的外部模块 | 字符串 | |
134+
| DATABASE_TYPE | 存储 flowise 数据的数据库类型 | 枚举字符串: `sqlite`, `mysql`, `postgres` | `sqlite` |
135+
| DATABASE_PATH | 数据库保存的位置（当 DATABASE_TYPE 是 sqlite 时） | 字符串 | `your-home-dir/.flowise` |
136+
| DATABASE_HOST | 主机 URL 或 IP 地址（当 DATABASE_TYPE 不是 sqlite 时） | 字符串 | |
137+
| DATABASE_PORT | 数据库端口（当 DATABASE_TYPE 不是 sqlite 时） | 字符串 | |
138+
| DATABASE_USERNAME | 数据库用户名（当 DATABASE_TYPE 不是 sqlite 时） | 字符串 | |
139+
| DATABASE_PASSWORD | 数据库密码（当 DATABASE_TYPE 不是 sqlite 时） | 字符串 | |
140+
| DATABASE_NAME | 数据库名称（当 DATABASE_TYPE 不是 sqlite 时） | 字符串 | |
141+
| SECRETKEY_PATH | 保存加密密钥（用于加密/解密凭据）的位置 | 字符串 | `your-path/Flowise/packages/server` |
142+
| FLOWISE_SECRETKEY_OVERWRITE | 加密密钥用于替代存储在 SECRETKEY_PATH 中的密钥 | 字符串 |
143+
| DISABLE_FLOWISE_TELEMETRY | 关闭遥测 | 字符串 |
144+
| MODEL_LIST_CONFIG_JSON | 加载模型的位置 | 字符 | `/your_model_list_config_file_path` |
145+
| STORAGE_TYPE | 上传文件的存储类型 | 枚举字符串: `local`, `s3` | `local` |
146+
| BLOB_STORAGE_PATH | 上传文件存储的本地文件夹路径, 当`STORAGE_TYPE`是`local` | 字符串 | `your-home-dir/.flowise/storage` |
147+
| S3_STORAGE_BUCKET_NAME | S3 存储文件夹路径, 当`STORAGE_TYPE`是`s3` | 字符串 | |
148+
| S3_STORAGE_ACCESS_KEY_ID | AWS 访问密钥 (Access Key) | 字符串 | |
149+
| S3_STORAGE_SECRET_ACCESS_KEY | AWS 密钥 (Secret Key) | 字符串 | |
150+
| S3_STORAGE_REGION | S3 存储地区 | 字符串 | |
151+
| S3_ENDPOINT_URL | S3 端点 URL | 字符串 | |
152+
| S3_FORCE_PATH_STYLE | 将其设置为 true 以强制请求使用路径样式寻址 | 布尔值 | false |
153+
| SHOW_COMMUNITY_NODES | 显示由社区创建的节点 | 布尔值 | |
154+
| DISABLED_NODES | 从界面中隐藏节点（以逗号分隔的节点名称列表） | 字符串 | |
156155

157156
您也可以在使用 `npx` 时指定环境变量。例如：
158157

Diff for: ‎package.json

+3
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,9 @@
1717
"start": "run-script-os",
1818
"start:windows": "cd packages/server/bin && run start",
1919
"start:default": "cd packages/server/bin && ./run start",
20+
"start-worker": "run-script-os",
21+
"start-worker:windows": "cd packages/server/bin && run worker",
22+
"start-worker:default": "cd packages/server/bin && ./run worker",
2023
"clean": "pnpm --filter \"./packages/**\" clean",
2124
"nuke": "pnpm --filter \"./packages/**\" nuke && rimraf node_modules .turbo",
2225
"format": "prettier --write \"**/*.{ts,tsx,md}\"",

0 commit comments

Comments
 (0)