Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Script engine audio listener #949

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ksuprynowicz
Copy link
Member

This will allow streaming in-game audio into script engine, where it can later be processed or sent to external speech-to-text system, for example through a websocket allowing things such as subtitles, automatic translation, and LLM-based chatbots that react to speech.

@ksuprynowicz ksuprynowicz added the work in progress Do not merge yet label May 3, 2024
@Armored-Dragon
Copy link
Member

I would love to have this feature for the automatic translation! No Mans Sky did this really well and it would be impressive to see this feature in a platform like Overte.

@ksuprynowicz
Copy link
Member Author

It already can be used for this on Linux :)
Together with R3ido we got it running with Collabora's WhisperLive, and this gives instantenous Speech To Text that can then be translated. I think Whisper also has own translation functionality?
If you are interested I can share scripts for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
work in progress Do not merge yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants