New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

关于扩充词表再增量预训练的疑问 #68

Open

qiancheng99 opened this issue May 19, 2023 · 0 comments

qiancheng99 commented May 19, 2023

Linly-Chinese-LLaMA-7B没有扩充词表。如果我自己训练了一个sentencepiece词表并和原版词表合并以后，要怎么改动代码扩充embedding进行预训练呢？
是否支持llama 13b的增量预训练呢？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment