Skip to content

Conversation

OrionMoonclaw
Copy link

This adds an ao driver that can be used to register a callback with libmpv to get raw PCM data. Useful for embedding mpv in applications such as game engines, which need the audio to be played in a 3D world and spatialized.

This was tested with libmpvbridge (not public yet), which I'm developing to use mpv in Renderite.Godot.

closes #9283

static bool set_pause(struct ao *ao, bool paused)
{
return true; // signal support so common code doesn't write silence
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so the client is not informed of a pause situation?

ao->bps = ao->channels.num * (int64_t)ao->samplerate * af_fmt_to_bytes(ao->format);

MP_INFO(ao, "libmpv: Samplerate: %d Hz Channels: %d Format: %s\n",
ao->samplerate, ao->channels.num, af_fmt_to_str(ao->format));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does the client find out which format the blob of data he gets is, anyway?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the audio current audio format can be grabbed as a property, though I suppose I could just pass that in the callback as well

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's also async, so won't work reliably.

I think it would be more practical if sample rate and channels were set ahead of time and then the driver would only support s16.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong, but it's already possible to set the format with this option before mpv initializes, so I think that should cover it. I might just add a note about it in the docs.

{"audio-format", OPT_AUDIOFORMAT(audio_output_format), .flags = UPDATE_AUDIO},

If that's not enough I could also add an option specific to this ao, I'd prefer to keep it configurable since I personally would like the samples as floats, it saves me an extra conversion step.


/**
* Set a custom function that should be called on new audio data. Raw PCM is passed in as an argument to the callback.
* This can only be used with ao set to libmpv and after the audio driver has been initialized.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

won't this mean that the client will potentially miss the first samples because he has to race against "ao init" until the callback can be set?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think registering the callback on MPV_EVENT_AUDIO_RECONFIG should be good enough, but I'll have a look at this later

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, because events are delivered async.

Copy link

github-actions bot commented Sep 7, 2025

@na-na-hi
Copy link
Contributor

na-na-hi commented Sep 8, 2025

IMO the libmpv AO initialization should block indefinitely until the client receives MPV_EVENT_AUDIO_RECONFIG (or add a dedicated event) and sets the callback. Subsequently, setting callback is not allowed until the AO is reinitialized again. This makes sure the first audio sample is processed by the callback, and runtime AO switch can be done correctly.

If the audio format can be customized, then the callback function should tell the client which format is in use, to avoid possible race conditions in accessing properties.

@OrionMoonclaw OrionMoonclaw marked this pull request as draft September 8, 2025 17:58
@OrionMoonclaw OrionMoonclaw force-pushed the ao_libmpv branch 2 times, most recently from 9da99e5 to 11c19d4 Compare September 8, 2025 18:06
@OrionMoonclaw
Copy link
Author

I marked it as a draft, since it's gonna need a bit more discussion.

Right now I made it so the callback can be set before ao init by storing it under MPContext, it feels a bit ugly, but it should work.

This adds an ao driver that can be used to register a callback with
libmpv to get raw PCM data. Useful for embedding mpv in applications
such as game engines, which need the audio to be played in a 3D world
and spatialized.

Signed-off-by: Orion Moonclaw <[email protected]>
mpctx, mpctx->encode_lavc_ctx, out_rate,
out_format, out_channels);

if (strcmp(mpctx->ao->driver->name, "libmpv") == 0)

Check failure

Code scanning / ClusterFuzzLite/CIFuzz

Don't crash Error

Null-dereference
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

libmpv: Get audio frame?

3 participants