You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Loading lora models has been implemented. However, it's not yet possible to save the merge result as a lora. While it wouldn't make sense to do this for full models, if the result happens to be in delta space, then we can apply the SVD algorithm on the appropriate keys. I was thinking we could add a new method merge_and_save_lora that only accepts a recipe that results in delta space. Like this, we can catch mistakes if a recipe doesn't do what it intends to and we can keep the current merge_and_save method as-is.
The text was updated successfully, but these errors were encountered:
Loading lora models has been implemented. However, it's not yet possible to save the merge result as a lora. While it wouldn't make sense to do this for full models, if the result happens to be in delta space, then we can apply the SVD algorithm on the appropriate keys. I was thinking we could add a new method
merge_and_save_lora
that only accepts a recipe that results in delta space. Like this, we can catch mistakes if a recipe doesn't do what it intends to and we can keep the currentmerge_and_save
method as-is.The text was updated successfully, but these errors were encountered: