VideoLingo is an all-in-one video translation, localization, and dubbing tool aimed at generating Netflix-quality subtitles. It eliminates stiff machine translations and multi-line subtitles while adding high-quality dubbing, enabling global knowledge sharing across language barriers.
Key features:
-
🎥 YouTube video download via yt-dlp
-
🎙️ Word-level subtitle recognition with WhisperX
-
📝 NLP and GPT-based subtitle segmentation
-
📚 GPT-generated terminology for coherent translation
-
🔄 3-step direct translation, reflection, and adaptation for professional-level quality
-
✅ Netflix-standard single-line subtitles only
-
🗣️ Dubbing alignment with GPT-SoVITS and other methods
-
🚀 One-click startup and output in Streamlit
-
📝 Detailed logging with progress resumption
-
🌐 Comprehensive multi-language support
Difference from similar projects: Single-line subtitles only, superior translation quality
ru_demo.mp4 |
sovits.mp4 |
OAITTS.mp4 |
Current input language support and examples:
| Input Language | Support Level | Translation Demo |
|---|---|---|
| English | 🤩 | English to Chinese |
| Russian | 😊 | Russian to Chinese |
| French | 🤩 | French to Japanese |
| German | 🤩 | German to Chinese |
| Italian | 🤩 | Italian to Chinese |
| Spanish | 🤩 | Spanish to Chinese |
| Japanese | 😐 | Japanese to Chinese |
| Chinese* | 🤩 | Chinese to English |
*Chinese requires separate configuration of the whisperX model, only applicable for local source code installation. See the installation documentation for the configuration process, and be sure to specify the transcription language as zh in the webpage sidebar
Translation language support depends on the capabilities of the large language model used, while dubbing language depends on the chosen TTS method.
Commercial version provides free 20min credits, visit videolingo.io
VideoLingo supports all hardware platforms and operating systems, but performs best with GPU acceleration. For detailed installation instructions , refer to the documentation: English | 简体中文
VideoLingo provides a Dockerfile. Refer to the installation documentation: English | 简体中文
Usage instructions: English | 简体中文
-
WhisperX performance varies across different devices. Version 1.7 performs demucs voice separation first, but this may result in worse transcription after separation compared to before. This is because whisper itself was trained in environments with background music - before separation it won't transcribe BGM lyrics, but after separation it might transcribe them.
-
The dubbing feature quality may not be perfect as it's still in testing and development stage, with plans to integrate MascGCT. For best results currently, it's recommended to choose TTS with similar speech rates based on the original video's speed and content characteristics. See the demo for effects.
-
Multilingual video transcription recognition will only retain the main language. This is because whisperX uses a specialized model for a single language when forcibly aligning word-level subtitles, and will delete unrecognized languages.
-
Multi-character separate dubbing is under development. While whisperX has VAD potential, specific implementation work is needed, and this feature is not yet supported.
- SaaS service at videolingo.io
- VAD to distinguish speakers, multi-character dubbing
- Customizable translation styles
- Lip sync for dubbed videos
This project is licensed under the Apache 2.0 License.The following open source projects provide important support for the development of VideoLingo:
whisperX | yt-dlp | json_repair | GPT-SoVITS | BELLE
- Join our Discord: https://discord.gg/9F2G92CWPp
- Submit Issues or Pull Requests on GitHub
- Follow me on Twitter: @Huanshere
- Email me at: [email protected]
If you find VideoLingo helpful, please give us a ⭐️!