Super high traffic usage when playing specific YouTube live stream

[Bakudankun]

mpv Information

mpv 0.38.0 Copyright © 2000-2024 mpv/MPlayer/mplayer2 projects
 built on Apr 20 2024 00:22:40
libplacebo version: v7.349.0 (v6.338.0-124-gd0eb997-dirty)
FFmpeg version: N-114897-gbba996d6c
FFmpeg library versions:
   libavutil       59.15.100
   libavcodec      61.5.103
   libavformat     61.3.100
   libswscale      8.2.100
   libavfilter     10.2.101
   libswresample   5.2.100

Other Information

• Windows version: Microsoft Windows [Version 10.0.22631.3737]
• GPU model, driver and version: Radeon 780M Graphics
• Source of mpv: Windows builds by shinchiro
• yt-dlp version: 2024.05.27

Reproduction Steps

1. Open Windows Resource Monitor to see network usage
2. mpv https://www.youtube.com/watch?v=thYDM-UAMKg
3. Wait a minute to see its traffic

Note: The stream lasts long but not permanent. In a few weeks, there may be no way to reproduce.

Expected Behavior

When played from a web browser e.g. chrome, its network usage is about 300KB per second (1080p).

Actual Behavior

The stream is played successfully, but mpv downloads over 6MB per second (not 6 megabits, it's 6 megabytes per second).

As far as I've tested, other streams do not have similar problem.

Log File

output.txt

Sample Files

No response

I carefully read all instruction and confirm that I did the following:

• I tested with the latest mpv version to validate that the issue is not already fixed.
• I provided all required information including system and mpv version.
• I produced the log file with the exact same set of files, parameters, and conditions used in "Reproduction Steps", with the addition of --log-file=output.txt.
• I produced the log file while the behaviors described in "Actual Behavior" were actively observed.
• I attached the full, untruncated log file.
• I attached the backtrace in the case of a crash.

[po5]

Use --ytdl-format. Default downloads the highest quality formats.

[Bakudankun]

@po5 Even playing lowest resolution video (144p) downloads about 3MB per second.
Again, playing the stream at the highest quality (1080p) with Chrome only downloads around 300KB per second.

[kasper93]

What is the problem exactly? I smell XY here...

Depending on your mpv cache options it will cache/download data as fast as possible, until the cache is filled.

It won't download more data than is needed for the playback, it will do it faster and stop the transfer quicker.

[Bakudankun]

@kasper93 No, this is mpv problem. Please read my report carefully. I beleave you can test on your side.
Resource Monitor shows average traffic for a minute. That means mpv downloads 360MB per minute to play a stream which is less than 3Mbps. Is it expected behavior? If so, what is the extra 340MB?
--no-cache does not solve the problem.

[Bakudankun]

You may not have noticed that is a live stream, so mpv starts playing from the current time.
There is no such amount of data to cache. Or, does mpv cache the future?

[kasper93]

You simply cannot take current average speed per second and extrapolate it to whatever duration you want. Note that resource monitor averages the values too.

Download https://learn.microsoft.com/en-us/sysinternals/downloads/process-explorer double click on mpv.exe and observe network tab. It will show you cumulative data received.

[Bakudankun]

@kasper93 I checked with procexp and saw the Receive Bytes increasing at a similar rate order (about 250MB per minute).
Can't you reproduce the problem? What information is missing?

[kasper93]

Feel free to report it to ffmpeg if there is a problem there, mpv doesn't control downloading rate/data.

[Bakudankun]

@kasper93 yt-dlp also utilizes ffmpeg, but it downloads in a reasonable rate.

I noticed that #EXT-X-PROGRAM-DATE-TIME in the mpv's log (in verbose mode) is 1 hour past from the current date & time.
When I fed ffmpeg the .m3u8 URL which mpv showed, the #EXT-X-PROGRAM-DATE-TIME in its log was also 1 hour past, and ffmpeg also loaded a few megabytes per second.
With yt-dlp, it was exact current date & time. With the .m3u8 which yt-dlp showed, ffmpeg also showed current time and load in reasonable rate (a few hundred kilobytes per second).

Maybe mpv loads the stream from the past 1 hour? How to let mpv to load the stream from current time?

[na-na-hi]

yt-dlp also utilizes ffmpeg, but it downloads in a reasonable rate.

yt-dlp only uses ffmpeg for muxing, not for networking. mpv uses ffmpeg for networking, and the issue is also reproducible with ffplay, so it's ffmpeg's HLS stream handling at fault.

(Side note: ffmpeg is bad at stream handling in general, especially HLS, and there are various bugs associated with it. VLC handles the link in the OP correctly without high traffic usage because it uses its own network layer. IMO mpv should also do the same, but for now you have to report the issue to ffmpeg.)

[Bakudankun]

Comparing both .m3u8 URLs, I noticed that mpv's URL has /playlist_type/DVR/ part while yt-dlp's has /playlist_type/LIVE/.
DVR on YouTube means the feature to rewind the stream. That may make mpv to load the live from the past.

Seeing the output of yt-dlp --dump-single-json, LIVE formats seem to have both video and audio.
So, I ran mpv with --ytdl-format=best (which specifies the best format with both video and audio), and now mpv's traffic is under 500KB/sec !

Why don't you make this behavior default? Is this still ffmpeg's problem??

Anyway, this problem has been solved on my end.
Thank you for your help!

[Bakudankun]

Why don't you make this behavior default?

Well, when playing other streams, there is no problem about download rate even when .m3u8 URL contains DVR part.
That may not always problematic. Please forget my suggestion.

[na-na-hi]

Why don't you make this behavior default?

--ytdl-format=best only selects a single format with both video and audio. For most videos this is only 720p30. It "works" for livestreams only because it selects a format from WEB which has both video and audio which doesn't trigger the ffmpeg bug. The default is to combine multiple formats with best video and audio from IOS.

Is this still ffmpeg's problem?

There is no reason why ffmpeg can't handle the mentioned stream properly.

[kasper93]

There is no reason why ffmpeg can't handle the mentioned stream properly.

I think we should keep this issue open for visibility.

yt-dlp also utilizes ffmpeg, but it downloads in a reasonable rate.

Like already said by na-na-hi, only for muxing. It would indeed be nice to improve that. I will reopen in case anyone wants to work on the improvments, either in ffmpeg or directly in mpv.

[MoSal]

This is very interesting since similar behavior showed up in my own stream grabber.

• This is a live HLS playlist with 1s long segments.
• This means you need to update the playlist every second to keep up with live updates.
• A single playlist download is >3MiB.
• So most of the traffic comes from playlist updates 🤣
• And this is why choosing a lower quality stream does not help.

Current HLS versions (might be still a draft) support DASH-style variable substitution, so this problem may go away in the future, but not anytime soon.

[MoSal]

I don't know why I didn't think of this yesterday, but here is a work-around:

--stream-lavf-o='http_seekable=0,headers=accept-encoding:gzip'

This makes use of the fact that HLS playlists are highly-compressible.
http_seekable=0 is needed because, without it, ffmpeg includes a Range: bytes=0- request header which causes the server to send a non-compressed reply.

[kasper93]

This should really be fixed on ffmpeg side or it cannot be used for modern content delivery services.

It is known issue for years: https://trac.ffmpeg.org/ticket/7158
Also ffmpeg supports only gzip.

[MoSal]

@kasper93

To be fair to ffmpeg, this is an unusual situation where the total duration of a live playlist is big, and the segment duration is small.

A segment duration of 5 seconds would reduce the overhead by ~25x, since you would have 1/5x of the segments in the playlist, and you would need to update the playlist 1/5x as frequently.

And a good client should allow controlling the frequency of playlist updates.

In my stream grabber, I already have a --playlist-req-freq=[0.1-1.0] option which is implemented like this:

    let playlist_req_freq = ctx.config.playlist_req_freq;
    let seg_avg_low_dur = (segs_dur / segs_count + min_seg_dur) / 2.0;

    // playlist_req_freq=0.1 => delay = seg_avg_low_dur*5
    // playlist_req_freq=0.5 => delay = seg_avg_low_dur
    // playlist_req_freq=1.0 => delay = seg_avg_low_dur*0.5
    let delay = seg_avg_low_dur / playlist_req_freq / 2.0;
    let delay = delay
        // in case the playlist is small (< 5 segments)
        .min(seg_avg_low_dur*segs_count)
        .max(1.0);

I thought this was enough, but in this use-case, the least frequent update period would still be 5 seconds. Better can be done, with taking the number of segments in the playlist into account.

As I already mentioned, variable substitution would help here too. So the problem may fix itself in X years 🥲.