Do not Work when I use multithreading on finetuning Lora Docowl 1.5.

[bhavdippatel]

When I am use multithreading and pass multiple question at same time it show the error of sentencepiece like out of index token id in my lora fine tuning. If anyone perform multithreading Please shere it how to solve this error.

[snow-like-kk]

Hi, can I ask for your flash-attn version ? When I run lora finrtuing script, some errors about flash-attn occur. I use CUDA11.7, torch 2.0.1 and flash-attn 2.3.5.

[bhavdippatel]

pip install torch=='2.4.1+cu121' torchvision=='0.19.1+cu121' torchaudio=='2.4.1+cu121' --index-url https://download.pytorch.org/whl/cu121

pip install icecream
pip install peft
pip install transformers==4.31.0
pip install flash_attn==2.6.3
pip install jsonlines
pip install deepspeed
(Colab pro+ A100)

[snow-like-kk]

pip install torch=='2.4.1+cu121' torchvision=='0.19.1+cu121' torchaudio=='2.4.1+cu121' --index-url https://download.pytorch.org/whl/cu121

pip install icecream pip install peft pip install transformers==4.31.0 pip install flash_attn==2.6.3 pip install jsonlines pip install deepspeed (Colab pro+ A100)

Thanks!