Hardware accelerated video with barely any cpu usage?

[markg85]

Hi,

These options allow you to play a video with hardware acceleration (if your pc has support for it through vaapi that is).

vo=gpu
hwdec=vaapi

In my case i have hardware acceleration support for (vainfo command output):

❯ vainfo
Trying display: wayland
Trying display: x11
vainfo: VA-API version: 1.21 (libva 2.21.0)
vainfo: Driver version: Mesa Gallium driver 24.1.1-arch1.1 for AMD Radeon Graphics (radeonsi, renoir, LLVM 17.0.6, DRM 3.57, 6.9.5-arch1-1)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointEncSlice
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc

Playback of a video (say h264, h265, vp9) just works fine and uses vaapi.
However, the CPU usage while playing is still ~10%.

When profiling this i stumble upon this:

Which seems to indicate copies to userspace and those taking up some percentage of cpu resources.
That makes me wonder, is there a combination of features in mpv that gives a "zerocopy" playback?
I expect zerocopy to have barely any cpu resources (at most a couple percent) as the heavy parts should be offloaded. Vaapi would handle the decoding and the gpu the presentation on screen.

[philipl]

10% of a core sounds very much expected. There's still plenty of work to do. And _copy_to_user is irrelevant to this discussion.

What you should do is compare your CPU usage with and without hwdec on and see how much you're saving. And yes, if you're decoding a low resolution clip in a simple codec, you won't be saving much. Compare 4K HEVC to see a real difference.

[llyyr]

Try profile=fast

mpv by default uses better rendering options, profile=fast will get you to parity with what VLC does.

[markg85]

Try profile=fast

mpv by default uses better rendering options, profile=fast will get you to parity with what VLC does.

I'm not sure what that is supposed to improve.
I tried it and what it does definitely faster is just starting up to begin with. Nice improvement for sure, but during playback it's just the same. There's hardly any difference, it remains mostly above 20%

[kasper93]

10% of a core sounds very much expected. There's still plenty of work to do. And _copy_to_user is irrelevant to this discussion.

That's surprising to me that you consider 10% normal. With hwdec and no other things like ASS subs on screen, I would expect CPU usage to be a lot lower, but I guess it depends of the CPU and its power state.

Either way, _copy_to_user seems to be relevant as it likely indicates dmabufs being cpu copied or something, which produces the bottleneck on @markg85 system.

I'm not sure what that is supposed to improve.

This only improves GPU load, not CPU one. In your case it looks like there is copyback going on between decoder and the renderer. Could you try Vulkan hwdec to establish some baseline?

vo=gpu-next
gpu-api=vulkan
hwdec=vulkan

Define RADV_PERFTEST=video_decode env before running mpv.

Also vaapi may work better with vo=gpu-next, you should try it too.

EDIT:

Are you sure you comparing those number at the same cpu power level / frequency? The % are not really comparable if it starts downclocing.

mpv cpu usage: Between 20 and 40%, hovering mostly around 24%
vlc cpu usage: Between 7 and 15%, hovering mostly around 11%

[markg85]

@kasper93 I too was surprised by the remarks of @philipl. CPU load, despite 10% being totally acceptable, is a thing and _copy_to_user definitely is relevant in that. While _copy_to_user itself might not be called directly by mpv, it is triggering a path where it is being called.

Let's forget the 10% and instead focus on ~24% which is the number for 4k playback.

Here's a new set of results:

• vaapi with vo=gpu-next: same with between 20 and 40%, hovering mostly around 24%
• vo=gpu-next, gpu-api=vulkan, hwdec=vulkan and RADV_PERFTEST=video_decode: surprisingly still the same as vaapi (20-40, mostly hovering around 24%)

mpv did say: Using hardware decoding (vulkan). so i'm assuming that vulkan is actually used to decode. I'm also assuming that it's just the video decode being done there and that the decoded frame is copied back to user space.

Regarding downclocking/frequency levels, etc.. That's a very good point! I wasn't expecting that to be a factor but i monitored it nonetheless. Turns out that for mpv my cpu is downclocked (and stays there) so the percentages of mpv are valid in that respect, just at a lower then expected frequency. It might be a bit lower when the cores are at full speed. However, not having it at max frequency shows these bottlenecks earlier so let's keep this one as-is.

For vlc... Things are different. The cores jump all around with the pc fan quickly starts spinning up because of it. It would probably have a higher cpu usage if i were to pin my cores in downclocked state.

Fact remains, in a proper hardware decoded pipeline there should be nearly no cpu usage of mpv at any frequency level :)

[kasper93]

Turns out that for mpv my cpu is downclocked (and stays there

That's means everything is working fine. You cannot compare usage % on downclocked CPU, they are very slow when in power saving states. So higher % is expected. And the fact CPU is downclocked makes it clear that mpv doesn't produce any significant load. Which we already seen at your perf profile, where only _copy_to_user is present and as we see now it is indeed neglectable background load.

However, not having it at max frequency shows these bottlenecks earlier so let's keep this one as-is.

No, you misunderstand this. There are no bottlenecks, your CPU is in lower powered mode already, there is no work to be done there. Usage % is not relevant.

For vlc... Things are different. The cores jump all around with the pc fan quickly starts spinning up because of it.

See, so there is more load when using VLC.

[markg85]

As another datapoint: ffplay in ffmpeg 7. ffplay didn't have it before this release but can, since v7, hardware decode on a vulkan surface. I have yet to compile this version but i'll definitely give it a shot!

[markg85]

RADV_PERFTEST=video_decode LD_LIBRARY_PATH=<path to compiled ffmpeg libs> ./ffplay -hwaccel vulkan ~/bbb_sunflower_2160p_60fps_normal.mp4

It surprisingly works!
CPU wise, it's - also surprisingly - around 24%!

It shows the same cpu usage characteristics as mpv.

[philipl]

Yeah. This means that the bulk of the time is either being spent in the driver or the common ffmpeg code - and if you were to sink enough time into your profiling, you'd be able to pull out enough information to show which it was.

Separately, you'd probably also observe a depressing amount of CPU being spent in your window manager/compositor that appears when you start playing and disappears when you stop.

But as @kasper93 says - these precentages aren't directly comparable if the CPU p-state isn't the same. You'd want to run with p-states disabled and the CPU running at a fixed speed to be able to truly compare. And there's probably also other CPU state you'd need to fix to be exactly equivalent - it's also so complex these days.

And for the record, when I said the CPU percentage was unsurprising, your numbers are well within the range that I've seen for hwdec over the years. You'd also see variation depending on the hardware in use (ie: drivers can use different amounts of CPU for equivalent functionality)

[markg85]

I'm sorry but i'm not buying that explanation. Call it a gut feeling. I might be completely wrong in the end. I'll do some more in depth benchmarks on my end.

For reference, just yesterday this blog hit the interwebs: https://www.supergoodcode.com/closing-the-loop/ (the url doesn't spoil it but it's about video encoding/decoding too).

And for the record, when I said the CPU percentage was unsurprising, your numbers are well within the range that I've seen for hwdec over the years. You'd also see variation depending on the hardware in use (ie: drivers can use different amounts of CPU for equivalent functionality)

I know. I never bothered to dive in deep but i do now.
Something feels off here, i just have to proof and point to that proof to make my point clear.
Or i'll be totally wrong and come back to tell that.

I'll be back :)

[kasper93]

I'm sorry but i'm not buying that explanation. Call it a gut feeling. I might be completely wrong in the end. I'll do some more in depth benchmarks on my end.

We are not selling. :) Do the benchmark, dissect the issue. You will get better understanding of the current behaviour, and maybe in the end you will find some tweaks. My argument is that benchmarking non fully cpu load, if always tricky, you get various synchronization points, you go through kernel to talk to device and so on. CPU relative load varies.