Merge branch 'main' of https://github.com/cgisky1980/ai00_rwkv_server

Ai00-X · Jul 25, 2023 · a4229c9 · a4229c9
2 parents 31ee93c + ca831de
commit a4229c9
Show file tree

Hide file tree

Showing 5 changed files with 505 additions and 292 deletions.
diff --git a/README.md b/README.md
@@ -1,164 +1,151 @@
 # 💯AI00 RWKV Server
-<!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
-[![All Contributors](https://img.shields.io/badge/all_contributors-4-orange.svg?style=flat-square)](#contributors-)
+
+<!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section --> 
+[![All Contributors](https://img.shields.io/badge/all_contributors-4-orange.svg?style=flat-square)](#contributors-) 
 <!-- ALL-CONTRIBUTORS-BADGE:END -->
 
-中文 | [English](README_en.md) | [日本語](README_jp.md)
+ [English](README.md) | [中文](README_zh.md)  | [日本語](README_jp.md)
 
 ---
-`AI00 RWKV Server`是一个基于[`RWKV`模型](https://github.com/BlinkDL/ChatRWKV)的推理API服务器。
-
-支持`VULKAN`推理加速，可以在所有支持`VULKAN`的GPU上运行。不用N卡！！！A卡甚至集成显卡都可加速！！！
-
-无需臃肿的`pytorch`、`CUDA`等运行环境，小巧身材，开箱即用！
 
-兼容OpenAI的ChatGPT API接口。
+`AI00 RWKV Server` is an inference API server based on the [`RWKV` model](https://github.com/BlinkDL/ChatRWKV).
 
-100% 开源可商用，采用MIT协议。
+It supports `VULKAN` inference acceleration and can run on all GPUs that support `VULKAN`. No need for Nvidia cards!!! AMD cards and even integrated graphics can be accelerated!!!
 
-如果您正在寻找一个快速、高效、易于使用的LLM API服务器，那么`AI00 RWKV Server`是您的最佳选择。它可以用于各种任务，包括聊天机器人、文本生成、翻译和问答。
+No need for bulky `pytorch`, `CUDA` and other runtime environments, it's compact and ready to use out of the box!
 
-立即加入`AI00 RWKV Server`社区，体验AI的魅力！
+Compatible with OpenAI's ChatGPT API interface.
 
-交流QQ群：30920262
+100% open source and commercially usable, under the MIT license.
 
-- [什么是 AI00](docs/ai00.md)
-- [为什么只支持RWKV](docs/rwkv.md)
+If you are looking for a fast, efficient, and easy-to-use LLM API server, then `AI00 RWKV Server` is your best choice. It can be used for various tasks, including chatbots, text generation, translation, and Q&A.
 
+Join the `AI00 RWKV Server` community now and experience the charm of AI!
 
-### 💥特色
+QQ Group for communication: 30920262
 
-- 基于`RWKV`模型，具有高性能和准确性
-- 支持`VULKAN`推理加速，不用该死的`CUDA`也能享受GPU加速！支持A卡、集成显卡等一切支持`VULKAN`的GPU
-- 无需臃肿的`pytorch`、`CUDA`等运行环境，小巧身材，开箱即用！
-- 兼容OpenAI的ChatGPT API接口
+### 💥Features
 
-### ⭕用途
+*   Based on the `RWKV` model, it has high performance and accuracy
+*   Supports `VULKAN` inference acceleration, you can enjoy GPU acceleration without the need for `CUDA`! Supports AMD cards, integrated graphics, and all GPUs that support `VULKAN`
+*   No need for bulky `pytorch`, `CUDA` and other runtime environments, it's compact and ready to use out of the box!
+*   Compatible with OpenAI's ChatGPT API interface
 
-- 聊天机器人
-- 文本生成
-- 翻译
-- 问答
-- 其他所有你能想到的LLM能干的事
+### ⭕Usages
 
-### 👻其他
+*   Chatbots
+*   Text generation
+*   Translation
+*   Q&A
+*   Any other tasks that LLM can do
 
-- 基于 [web-rwkv](https://github.com/cryscan/web-rwkv) 项目
-- [模型下载](https://huggingface.co/cgisky/RWKV-safetensors-fp16)
+### 👻Other
 
-## 安装、编译和使用
+*   Based on the [web-rwkv](https://github.com/cryscan/web-rwkv) project
+*   [Model download](https://huggingface.co/cgisky/RWKV-safetensors-fp16)
 
-### 📦直接下载安装
+## Installation, Compilation, and Usage
 
-1. 直接从 [Release](https://github.com/cgisky1980/ai00_rwkv_server/releases) 下载最新版本
+### 📦Direct Download and Installation
 
-2. [下载模型](https://huggingface.co/cgisky/RWKV-safetensors-fp16)后把模型放在`assets/models/`路径，例如`assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st`
-
-3. 在命令行运行
-
-    ```bash     
+1.  Directly download the latest version from [Release](https://github.com/cgisky1980/ai00_rwkv_server/releases)
+
+2.  After [downloading the model](https://huggingface.co/cgisky/RWKV-safetensors-fp16), place the model in the `assets/models/` path, for example, `assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st`
+
+3.  Run in the command line
+
+    ```bash
     $ ./ai00_rwkv_server --model assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st
     ```
-4. 打开浏览器，访问WebUI
-   [`http://127.0.0.1:3000`](http://127.0.0.1:3000)
-
-### 📜从源码编译
-
-1. [安装Rust](https://www.rust-lang.org/)
+
+4.  Open the browser and visit the WebUI [`http://127.0.0.1:3000`](http://127.0.0.1:3000)
+
 
-2. 克隆本仓库
+### 📜Compile from Source Code
 
+1.  [Install Rust](https://www.rust-lang.org/)
+
+2.  Clone this repository
+
     ```bash
-    $ git clone https://github.com/cgisky1980/ai00_rwkv_server.git
-    $ cd ai00_rwkv_server
+    $ git clone https://github.com/cgisky1980/ai00_rwkv_server.git $ cd ai00_rwkv_server
     ```
 
-
-3. [下载模型](https://huggingface.co/cgisky/RWKV-safetensors-fp16)后把模型放在
-`assets/models/`路径下，例如`assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st`
-
-4. 编译
-
+3.  After [downloading the model](https://huggingface.co/cgisky/RWKV-safetensors-fp16), place the model in the `assets/models/` path, for example, `assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st`
+
+4.  Compile
+
     ```bash
     $ cargo build --release
     ```
-     
-
-5. 编译完成后运行
-
-    ```bash     
-    $ cargo run --release -- --model assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st 
+
+5.  After compilation, run
+
+    ```bash
+    $ cargo run --release -- --model assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st
     ```
-
-6. 打开浏览器，访问WebUI
-   [`http://127.0.0.1:3000`](http://127.0.0.1:3000)
-
 
-## 📝支持的启动参数
-- `--model`: 模型路径
-- `--tokenizer`: 词表路径
-- `--port`: 运行端口
+6.  Open the browser and visit the WebUI [`http://127.0.0.1:3000`](http://127.0.0.1:3000)
+
 
+## 📝Supported Arguments
 
-## 📙目前可用的API
+*   `--model`: Model path
+*   `--tokenizer`: Tokenizer path
+*   `--port`: Running port
 
-API 服务开启于 3000 端口, 数据输入已经输出格式遵循Openai API 规范。
+## 📙Currently Available APIs
 
-- `/v1/chat/completions`
-- `/chat/completions`
-- `/v1/completions`
-- `/completions`
-- `/v1/embeddings`
-- `/embeddings`
+The API service starts at port 3000, and the data input and output format follow the Openai API specification.
 
-## 📙WebUI 截图
+*   `/v1/chat/completions`
+*   `/chat/completions`
+*   `/v1/completions`
+*   `/completions`
+*   `/v1/embeddings`
+*   `/embeddings`
+
+## 📙WebUI Screenshots
 
 ![image](https://github.com/cgisky1980/ai00_rwkv_server/assets/82481660/33e8da0b-5d3f-4dfc-bf35-4a8147d099bc)
 
 ![image](https://github.com/cgisky1980/ai00_rwkv_server/assets/82481660/a24d6c72-31a0-4ff7-8a61-6eb98aae46e8)
 
-
 ## 📝TODO List
 
-- [x] 支持`text_completions`和`chat_completions`
-- [x] 支持`sse`推送
-- [x] 添加`embeddings`
-- [x] 集成基本的调用前端
-- [ ] `Batch serve`并行推理
-- [ ] `int8`量化支持
-- [ ] `SpQR`量化支持
-- [ ] `LoRA`模型支持
-- [ ] `LoRA`模型热加载、切换
+*   [x] Support for `text_completions` and `chat_completions`
+*   [x] Support for sse push
+*   [x] Add `embeddings`
+*   [x] Integrate basic front-end
+*   [ ] Parallel inference via `batch serve`
+*   [ ] Support for `int8` quantization
+*   [ ] Support for `SpQR` quantization
+*   [ ] Support for `LoRA` model
+*   [ ] Hot loading and switching of `LoRA` model
 
 ## 👥Join Us
 
-我们一直在寻找有兴趣帮助我们改进项目的人。如果你对以下任何一项感兴趣，请加入我们！
-
-- 💀编写代码
-- 💬提供反馈
-- 🔆提出想法或需求
-- 🔍测试新功能
-- ✏翻译文档
-- 📣推广项目
-- 🏅其他任何会对我们有所帮助的事
+We are always looking for people interested in helping us improve the project. If you are interested in any of the following, please join us!
 
-无论你的技能水平如何，我们都欢迎你加入我们。你可以通过以下方式加入我们：
+*   💀Writing code
+*   💬Providing feedback
+*   🔆Proposing ideas or needs
+*   🔍Testing new features
+*   ✏Translating documentation
+*   📣Promoting the project
+*   🏅Anything else that would be helpful to us
 
-- 加入我们的 Discord 频道
-- 加入我们的 QQ 群
-- 在 GitHub 上提交问题或拉取请求
-- 在我们的网站上留下反馈
-
-我们迫不及待地想与你合作，让这个项目变得更好！希望项目对你有帮助！
+No matter your skill level, we welcome you to join us. You can join us in the following ways:
 
-## Thanks
+*   Join our Discord channel
+*   Join our QQ group
+*   Submit issues or pull requests on GitHub
+*   Leave feedback on our website
 
+We can't wait to work with you to make this project better! We hope the project is helpful to you!
 
-[![cryscan](https://avatars.githubusercontent.com/u/16053640?s=32&v=4)](https://github.com/cryscan)
-感谢cryscan的辛勤付出，为项目做出了杰出的贡献。
-
-### 感谢下面项目的编写者们做出的杰出工作
 
+## Thank you to these awesome individuals who are insightful and outstanding for their support and selfless dedication to the project
 <!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
 <!-- prettier-ignore-start -->
 <!-- markdownlint-disable -->
@@ -179,24 +166,6 @@ API 服务开启于 3000 端口, 数据输入已经输出格式遵循Openai API
 <!-- ALL-CONTRIBUTORS-LIST:END -->
 
 
-
-
-### 感谢下面又好看又有眼光又优秀的杰出人士对项目的支持和无私奉献
-
-- 来自 QQ 群
-
-    ![image](https://github.com/cgisky1980/ai00_rwkv_server/assets/82481660/6e324617-6d0c-49fd-ab1e-fd9cf02df51e)
-
-- 来自 Github 
-
-- 来自 Discord
-
-我们很感激您的帮助，我们很高兴能与您合作。
-
-
 ## Stargazers over time
 
 [![Stargazers over time](https://starchart.cc/cgisky1980/ai00_rwkv_server.svg)](https://starchart.cc/cgisky1980/ai00_rwkv_server)
-
-
-