Replies: 1 comment 3 replies
-
Made some progress: had to convert the model using llama.cpp script
However got the error:
Can I update some parameter in my |
Beta Was this translation helpful? Give feedback.
-
Made some progress: had to convert the model using llama.cpp script
However got the error:
Can I update some parameter in my |
Beta Was this translation helpful? Give feedback.
-
I have finetuned OpenHermes 2.5 using axolotl library, with qlora method and have got a bin file. Now I am trying to run the server with the lora adapter loaded, but it is erroring out with the following message:
llama_apply_lora_from_file_internal: unsupported file version.
Here is the log portion that shows up on the screen:
Here is the command line I am using:
How can I host the model with adapter using server? Does the adapter need to be in gguf as well? This link has an example where the main model is gguf and adapter is a bin file, so I tried it out.
Do I need any conversion of the adapter .bin file post the finetuning step, before running with llama.cpp server? I do not want to merge immediately before testing, and possibly retain multiple adapters in separate folders alongwith base model.
Thanks for any pointers.
Beta Was this translation helpful? Give feedback.
All reactions