Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

qwen2.5-0.5B作为基座使用flagembedding-decoder_only脚本训练embedding模型 #1376

Open
hellostronger opened this issue Feb 18, 2025 · 1 comment
Open

qwen2.5-0.5B作为基座使用flagembedding-decoder_only脚本训练embedding模型 #1376

hellostronger opened this issue Feb 18, 2025 · 1 comment

Comments

@hellostronger
Copy link

Image
这个是我的训练脚本，同一分数据，我微调zh-v1.5-bge-small，在验证集效果更好，为什么qwen0.5B参数量不是更大吗，求指点哪里出了问题
@545999961
Copy link
Collaborator

545999961 commented Feb 20, 2025 •
edited
Loading

数据量是多少呢
zh-v1.5-bge-small已经具备了通用的检索能力，只需要在领域内稍微训一下就可以达到很好的效果了
而qwen0.5B本身是不具备检索能力的，因此微调qwen0.5B需要足够多的数据

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@hellostronger @545999961