The Whisper plugin is an implementation of the Whisper speech recognition model developed by OpenAI as a Vamp plugin. The Whisper plugin analyses the text in the audio stream and generates markers corresponding to the tokens (words and/or syllables) found. The lightweight ggml-tiny model is embedded in the plugin (so you don’t have to download anything to start experimenting), but it is possible to download and use other models that may be more appropriate to your needs.
The Whisper Vamp Plugin has been designed for use in the free audio analysis application Partiels.
Download the Whisper Vamp plugin installation package for your operating system from the Releases section and run the installer.
Launch the Partiels application. In a new or existing document, create a new analysis track with the Whisper plugin. Modify the model or the analysis parameters via the property window. Please refer to the manual available in the Releases section for further information.
The compilation system is based on CMake, for example:
cmake . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build
ctest -C Debug -VV --test-dir build
- Whisper Vamp plugin by Pierre Guillot at IRCAM IMR Department
- Whisper.cpp by Georgi Gerganov
- Whisper model by OpenAI
- Vamp SDK by Chris Cannam, copyright (c) 2005-2024 Chris Cannam and Centre for Digital Music, Queen Mary, University of London.
- Ircam Vamp Extension by Pierre Guillot at IRCAM IMR department.