Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hubert and non english #67

Open
patriotyk opened this issue Jan 9, 2025 · 1 comment
Open

Hubert and non english #67

patriotyk opened this issue Jan 9, 2025 · 1 comment

Comments

@patriotyk
Copy link

In your readme you say that Hubert is useful for non english language, but Hubert model used in the script, trained only on English audio. Do you mean I need to replace hubert model with trained on my language(which doesn't exist, so I need to train it myself)? Also why do you use finetuned for ASR model instead of just pretrained.

@Fictionarry
Copy link
Owner

In our test, the English-pretrained HuBERT is general for most non-English cases like Chinese, French, and German, so you don't need to do anything extra but just apply it if you can't find a more suitable model. The model fine-tuned for ASR is better at phoneme alignment, which is an important factor for talking head, while the raw pre-trained model performs worse at this aspect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants