Skip to content

Commit

Permalink
Move Config.toml.
Browse files Browse the repository at this point in the history
  • Loading branch information
cryscan committed Oct 12, 2023
1 parent 0893010 commit ebf8436
Show file tree
Hide file tree
Showing 5 changed files with 13 additions and 13 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ QQ Group for communication: 30920262

2. After [downloading the model](https://huggingface.co/cgisky/RWKV-safetensors-fp16), place the model in the `assets/models/` path, for example, `assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st`

3. Optionally modify `Config.toml` for model configurations like model path, quantization layers, etc.
3. Optionally modify [`assets/Config.toml`](./assets/Config.toml) for model configurations like model path, quantization layers, etc.

4. Run in the command line

Expand Down Expand Up @@ -106,7 +106,7 @@ QQ Group for communication: 30920262

## 📝Supported Arguments

* `--model`: Model configure file path (default: `Config.toml`)
* `--config`: Configure file path (default: `assets/Config.toml`)
* `--tokenizer`: Tokenizer path
* `--port`: Running port
* `--adapter`: Adapter (GPU and backend) selection options: `Auto` and `Manual`
Expand All @@ -117,7 +117,7 @@ QQ Group for communication: 30920262
The server listens on port 3000, loads the full-layer quantized (32 > 24) 0.4B model, and selects the high-performance adapter.

```bash
$ cargo run --release -- --model Config.toml --port 3000 --adapter auto
$ cargo run --release -- --model assets/Config.toml --port 3000 --adapter auto
```

## 📙Currently Available APIs
Expand Down
6 changes: 3 additions & 3 deletions README_jp.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ OpenAIのChatGPT APIインターフェースと互換性があります。

2. [モデルをダウンロード](https://huggingface.co/cgisky/RWKV-safetensors-fp16)し、`assets/models/`パスに配置します。例:`assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st`

3. 必要に応じて、モデルのパスや量子化レイヤーなどのモデル設定のために Config.toml を修正します。
3. 必要に応じて、モデルのパスや量子化レイヤーなどのモデル設定のために [`assets/Config.toml`](./assets/Config.toml) を修正します。

4. コマンドラインで実行します

Expand Down Expand Up @@ -94,7 +94,7 @@ OpenAIのChatGPT APIインターフェースと互換性があります。

## 📝サポートされている起動パラメーター

* `--model`: モデル設定ファイルのパス(既定:`Config.toml`
* `--config`: モデル設定ファイルのパス(既定:`assets/Config.toml`
* `--tokenizer`: トークナイザーのパス
* `--port`: 実行ポート
* `--quant`: 量子化レイヤーの数を指定
Expand All @@ -106,7 +106,7 @@ OpenAIのChatGPT APIインターフェースと互換性があります。
サーバーはポート3000でリッスンし、全レイヤー量子化(32 > 24)の0.4Bモデルをロードし、高性能アダプターの自動選択。

```bash
$ cargo run --release -- --model Config.toml --port 3000 --adapter auto
$ cargo run --release -- --model assets/Config.toml --port 3000 --adapter auto
```

## 📙現在利用可能なAPI
Expand Down
6 changes: 3 additions & 3 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@

2. [下载模型](https://huggingface.co/cgisky/RWKV-safetensors-fp16)后把模型放在`assets/models/`路径,例如`assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st`

3. 你可以修改`Config.toml`里面的模型配置,包括模型路径、量化层数等
3. 你可以修改 [`assets/Config.toml`](./assets/Config.toml) 里面的模型配置,包括模型路径、量化层数等

4. 在命令行运行

Expand Down Expand Up @@ -102,7 +102,7 @@


## 📝支持的启动参数
- `--model`: 模型配置文件路径(默认`Config.toml`
- `--config`: 模型配置文件路径(默认`assets/Config.toml`
- `--tokenizer`: 词表路径
- `--port`: 运行端口
- `--adapter`: 适配器(GPU和后端)选择项:`Auto``Manual`
Expand All @@ -112,7 +112,7 @@

服务器监听3000端口,加载全部层量化(32 > 24)的0.4B模型,自动选择高性能适配器。
```bash
$ cargo run --release -- --model Config.toml --port 3000 --adapter auto
$ cargo run --release -- --model assets/Config.toml --port 3000 --adapter auto
```


Expand Down
File renamed without changes.
8 changes: 4 additions & 4 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -605,7 +605,7 @@ struct Args {
#[arg(long, short, value_name = "FILE")]
tokenizer: Option<PathBuf>,
#[arg(long, short, value_name = "FILE")]
model: Option<PathBuf>,
config: Option<PathBuf>,
#[arg(long, short)]
ip: Option<Ipv4Addr>,
#[arg(long, short, default_value_t = 65530)]
Expand Down Expand Up @@ -648,10 +648,10 @@ async fn main() {
let (sender, receiver) = flume::unbounded::<ThreadRequest>();

{
let path = args.model.clone().unwrap_or("Config.toml".into());
log::info!("loading model config {}...", path.to_string_lossy());
let path = args.config.clone().unwrap_or("assets/Config.toml".into());
log::info!("reading config {}...", path.to_string_lossy());

let request = reload_request_from_config(path).expect("load model config failed");
let request = reload_request_from_config(path).expect("load config failed");
let _ = sender.send(ThreadRequest::Reload(request));
}

Expand Down

0 comments on commit ebf8436

Please sign in to comment.