You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
target_modules: List of module names or regex expression of the module names to replace with Lora. target_modules=['query', 'value'],应该使用完整的参数名称或者正则表达式,如target_modules=['*query*', '*value*'], 可参考target_module写法
请提出你的问题
model=ppnlp.transformers.NeZhaForSequenceClassification.from_pretrained('nezha-large-wwm-chinese', num_classes=14)
tokenizer = ppnlp.transformers.NeZhaTokenizer.from_pretrained('nezha-large-wwm-chinese')
lora_config = LoRAConfig(
r=16,
target_modules=['query', 'value'],
lora_alpha=32,
lora_dropout=0.05,
)
lora_model = LoRAModel(model, lora_config)
model = lora_model
运行后显示[2024-06-21 21:58:38,253] [ INFO] - Frozen parameters: 0.00e+00 || Trainable parameters:3.25e+08 || Total parameters:3.25e+08|| Trainable:100.00%
LoRA没有起作用是为什么呢?以上写法的问题在哪
The text was updated successfully, but these errors were encountered: