Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Whisper support #686

Closed
jaggzh opened this issue May 18, 2023 · 5 comments
Closed

Whisper support #686

jaggzh opened this issue May 18, 2023 · 5 comments

Comments

@jaggzh
Copy link

jaggzh commented May 18, 2023

OpenAI-Whisper is a really high quality locally-run speech-to-text project. (It can also translate; but nevermind that for now).

It would be awesome if we could get that in here, even for the trigger word recognition.
I started a separate project to use it for live transcription. Whisper runs as a single command, or from python. My project includes a little python flask app that keeps the model loaded, takes a wav file (POSTed), and returns just the text.
https://github.com/jaggzh/whisperpluck

Before I do any effort to implement it in Kalliope I wanted to run it by you all. (I'm quite busy so I don't think I'll even be able to get to it for now. I coded whisperpluck for a friend who has issues using the computer).

@joshuaboniface
Copy link
Contributor

#687 adds this!

@joshuaboniface
Copy link
Contributor

#687 was merged, so the next release should have (core) Whisper support.

Next step is FasterWhisper, which is indeed much faster. This is waiting on the upstream PR Uberi/speech_recognition#693 to implement in speech_recognition, but otherwise I have the code ready. If you want to test those out, the branch blkal of both Kalliope and Speech Recognition in my respective forks have my latest code for both.

@jaggzh
Copy link
Author

jaggzh commented Jul 2, 2023

I'm listening
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/home/jaggz/venvs/kalliope/lib/python3.11/site-packages/kalliope/core/OrderListener.py", line 52, in run
    self.stt_instance = self.load_stt_plugin()
                        ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jaggz/venvs/kalliope/lib/python3.11/site-packages/kalliope/core/OrderListener.py", line 67, in load_stt_plugin
    return Utils.get_dynamic_class_instantiation(package_name='stt',
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jaggz/venvs/kalliope/lib/python3.11/site-packages/kalliope/core/Utils/Utils.py", line 135, in get_dynamic_class_instantiation
    mod = __import__(package_path, fromlist=[module_name.capitalize()])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'kalliope.stt.whisper'
^C

Ctrl+C pressed. Killing Kalliope

@jaggzh
Copy link
Author

jaggzh commented Jul 2, 2023

Oh, this happens after I say Kaliope (during STT?)

@jaggzh
Copy link
Author

jaggzh commented Jul 2, 2023

Oh, sorry, I was in master; I noticed in your PR it's in dev. That's working. :))
Closing this issue.

@jaggzh jaggzh closed this as completed Jul 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants