Skip to content

Commit

Permalink
fix adaptor
Browse files Browse the repository at this point in the history
  • Loading branch information
cgisky1980 committed Aug 25, 2023
1 parent e311332 commit 6e35848
Show file tree
Hide file tree
Showing 4 changed files with 8 additions and 8 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,14 +97,14 @@ QQ Group for communication: 30920262
* `--tokenizer`: Tokenizer path
* `--port`: Running port
* `--quant`: Specify the number of quantization layers
* `--adepter`: Adapter (GPU and backend) selection options
* `--adaptor`: Adapter (GPU and backend) selection options

### Example

The server listens on port 3000, loads the full-layer quantized (32 > 24) 0.4B model, and selects adapter 0 (to get the specific adapter number, you can first not add this parameter, and the program will enter the adapter selection page).

```bash
$ cargo run --release -- --model assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st --port 3000 --quant 32 --adepter 0
$ cargo run --release -- --model assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st --port 3000 --quant 32 --adaptor 0
```

## 📙Currently Available APIs
Expand Down
4 changes: 2 additions & 2 deletions README_jp.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,14 +95,14 @@ OpenAIのChatGPT APIインターフェースと互換性があります。
* `--tokenizer`: トークナイザーのパス
* `--port`: 実行ポート
* `--quant`: 量子化レイヤーの数を指定
* `--adepter`: アダプター(GPUおよびバックエンド)の選択オプション
* `--adaptor`: アダプター(GPUおよびバックエンド)の選択オプション

###

サーバーはポート3000でリッスンし、全レイヤー量子化(32 > 24)の0.4Bモデルをロードし、アダプター0を選択します(特定のアダプター番号を取得するには、最初にこのパラメーターを追加せず、プログラムがアダプター選択ページに入るまで待ちます)。

```bash
$ cargo run --release -- --model assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st --port 3000 --quant 32 --adepter 0
$ cargo run --release -- --model assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st --port 3000 --quant 32 --adaptor 0
```

## 📙現在利用可能なAPI
Expand Down
4 changes: 2 additions & 2 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,13 +104,13 @@
- `--tokenizer`: 词表路径
- `--port`: 运行端口
- `--quant`: 指定量化层数
- `--adepter`: 适配器(GPU和后端)选择项
- `--adaptor`: 适配器(GPU和后端)选择项

### 示例

服务器监听3000端口,加载全部层量化(32 > 24)的0.4B模型,选择0号适配器(要查看具体适配器编号可以先不加该参数,程序会先进入选择页面)。
```bash
$ cargo run --release -- --model assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st --port 3000 --quant 32 --adepter 0
$ cargo run --release -- --model assets/models/RWKV-4-World-0.4B-v1-20230529-ctx4096.st --port 3000 --quant 32 --adaptor 0
```


Expand Down
4 changes: 2 additions & 2 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -351,7 +351,7 @@ fn model_task(model: Model, tokenizer: Tokenizer, receiver: Receiver<ThreadReque
#[command(author, version, about, long_about = None)]
struct Args {
#[arg(long, short)]
adepter: Option<usize>,
adaptor: Option<usize>,
#[arg(long, short, value_name = "FILE")]
model: Option<String>,
#[arg(long, short, value_name = "FILE")]
Expand Down Expand Up @@ -390,7 +390,7 @@ async fn main() -> Result<()> {
);

let (sender, receiver) = flume::unbounded::<ThreadRequest>();
let env = create_environment(args.adepter).await?;
let env = create_environment(args.adaptor).await?;
let tokenizer = load_tokenizer(&tokenizer_path)?;

log::info!("{:#?}", env.adapter.get_info());
Expand Down

0 comments on commit 6e35848

Please sign in to comment.