Cannot load vicuna-7b-delta-v0

[LianxinGao]

1. Use aspen.load_llama_tf_weight to load vicuna-7b-delta-v0 model use more than 30GB memory then caused OOM.

2. Use utils.convert_hf_to_pth to transfer vicuna-7b-delta-v0 to .pth model, then use aspen.load_llama_7b_weight to load .pth model, an error is reported:

Not use layer model.embed_tokens.weight.
Traceback (most recent call last):
  File "/data/glx/code/multi_lora/legacy.py", line 43, in <module>
    aspen.load_llama_7b_weight(llama_model, config["base_model"], config["device"])
  File "/data/glx/code/multi_lora/aspen/modelloader.py", line 21, in load_llama_7b_weight
    layer_id = int(layer_name[:layer_name.find(".")])
ValueError: invalid literal for int() with base 10: 'ayers'

[github-actions]

welcome to the Aspen' first issue

[mikecovlee]

模型是未恢复的delta还是已经恢复了的完整weight？如果只是delta是肯定不能加载的，你可以试一下mlora.py仅加载模型有没有问题

…

---- Replied Message ---- | From | ***@***.***> | | Date | 08/29/2023 22:03 | | To | TUDB-Labs/multi-lora-fine-tune ***@***.***> | | Cc | Subscribed ***@***.***> | | Subject | Re: [TUDB-Labs/multi-lora-fine-tune] Cannot load vicuna-7b-delta-v0 (Issue #14) | welcome to the Aspen' first issue — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[mikecovlee]

另外可以试一下mikecovlee_dev分支中支持8bit量化的版本

…

---- Replied Message ---- | From | ***@***.***> | | Date | 08/29/2023 22:03 | | To | TUDB-Labs/multi-lora-fine-tune ***@***.***> | | Cc | Subscribed ***@***.***> | | Subject | Re: [TUDB-Labs/multi-lora-fine-tune] Cannot load vicuna-7b-delta-v0 (Issue #14) | welcome to the Aspen' first issue — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[mikecovlee]

噢，我知道咋回事了，明天我推个patch

[mikecovlee]

Plz close this issue if problem solved.