Skip to content

Commit 9026688

Browse files
authored
Update README.md
1 parent 5300d32 commit 9026688

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

README.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@
66

77
After installation, you can access to `hip` package from any project. `hip` is code name of HiP attention.
88

9+
[vLLM Implementation](https://github.com/DeepAuto-AI/vllm)
10+
911
## TL;DR
1012

1113
**We provide OpenAI Compatible API server built with vLLM and HiP attention!**
@@ -284,4 +286,4 @@ python examples/openai_chat_image_client.py --image-file="images/cherry_blossom.
284286
## Example training command
285287
```bash
286288
OMP_NUM_THREADS=16 CUDA_VISIBLE_DEVICES=0,1,2,3 PYTHONPATH=. accelerate launch --num_processes=4 --main_process_port 29501 hip/trainer/hip_trainer_hf.py --method hip --block_size_q 32 --block_size_k 2 --k 512 --lora_r 256 --dataset openwebtext --dense_layers 4 --name bs16_warmup10_dq2k --dense_queries 2048 --seq_len 32768 --disable_kd --sparsity_reg 0.01 --gradient_accumulation_steps 4 --warmup_steps 10 --model giraffe13b --using_deepspeed
287-
```
289+
```

0 commit comments

Comments
 (0)