Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qlora微调之后的模型为什么不能merge lora #3320

Open
ArlanCooper opened this issue Feb 28, 2025 · 0 comments
Open

qlora微调之后的模型为什么不能merge lora #3320

ArlanCooper opened this issue Feb 28, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@ArlanCooper
Copy link

Describe the bug
What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程,最好有截图)
以前版本的ms-swift,是可以进行merge lora的,为什么现在版本的不可以了呢

之前的脚本:

# Experimental environment: 3090, A10, V100
# 20GB GPU memory
CUDA_VISIBLE_DEVICES=1 \
swift sft \
    --model_type qwen2_5-72b-instruct \
    --model_id_or_path /data/share/rwq/Qwen2.5-72B-Instruct \
    --model_revision master \
    --sft_type lora \
    --tuner_backend peft \
    --template_type AUTO \
    --dtype AUTO \
    --output_dir output_qwen2 \
    --train_dataset_sample -1 \
    --num_train_epochs 8 \
    --max_length 3600 \
    --check_dataset_strategy warning \
    --quantization_bit 4 \
    --bnb_4bit_comp_dtype AUTO \
    --lora_rank 8 \
    --lora_alpha 32 \
    --lora_dropout_p 0.05 \
    --lora_target_modules ALL \
    --gradient_checkpointing true \
    --batch_size 1 \
    --weight_decay 0.1 \
    --learning_rate 4e-5 \
    --gradient_accumulation_steps 16 \
    --max_grad_norm 0.5 \
    --warmup_ratio 0.03 \
    --eval_steps 100 \
    --save_steps 100 \
    --save_total_limit 2 \
    --logging_steps 10 \
    --use_flash_attn true \
 


是可以使用qlora进行merge的

现在版本的qlora微调脚本:

CUDA_VISIBLE_DEVICES=3 \
swift sft \
    --model /home/octopus/data/llm_list/DeepSeek-R1-Distill-Llama-70B \
    --train_type lora \
    --torch_dtype bfloat16 \
    --bnb_4bit_compute_dtype bfloat16 \
    --bnb_4bit_quant_type nf4 \
    --bnb_4bit_use_double_quant true \
    --quant_method bnb \
    --quant_bits 4 \
    --num_train_epochs 8 \
    --per_device_train_batch_size 1 \
    --per_device_eval_batch_size 1 \
    --learning_rate 4e-5 \
    --lora_rank 8 \
    --lora_alpha 32 \
    --target_modules all-linear \
    --gradient_accumulation_steps 16 \
    --eval_steps 100 \
    --save_steps 100 \
    --save_total_limit 2 \
    --logging_steps 5 \
    --max_length 20000 \
    --output_dir output_Llama \
    --warmup_ratio 0.05 \
    --dataloader_num_workers 4 \
    --model_author swift \
    --model_name swift-robot \
    --attn_impl flash_attn \

Your hardware and system info
Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息,如CUDA版本,系统,GPU型号和torch版本等)
ubutu 14.04
torch 2.4.0+cu121
GPU A100

Additional context
Add any other context about the problem here(在这里补充其他信息)

@Jintao-Huang Jintao-Huang added the enhancement New feature or request label Feb 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants