Count decoded pixels.

[j0sh]

• Adds support for counting decoded pixels
• Allows passing in empty output parameters to force a decode-only flow.

The GetMediaInfo function in go-livepeer becomes:

func GetMediaInfo(fname string) (*MediaInfo, error) {
       in := &TranscodeOptionsIn{Fname: fname}
       res, err := Transcode3(in, nil)
       if err != nil {
               return nil, err
       }
       return &res.Decoded, nil
}

This could be further specialized to return pixels only, rather than a generic struct, because go-livepeer not likely to need the frame count.

Fixes #125

[j0sh]

Ready for review

[darkdarkdragon]

LGTM

[darkdarkdragon]

@j0sh @yondonfu If I got it right, this will be called by broadcaster for every transcoded segment? So broadcaster will be doing software decoding of every transcoder stream? This will significantly increase CPU requirements, I'll need to re-test broadcaster's throughput with this turned on.

[yondonfu]

LGTM 🚢

[yondonfu]

@darkdarkdragon Yeah the current plan is for B to check the pixel count for each transcoded result. This will be async (see this discussion thread). I'm interested in how this impacts the benchmarking results you've been doing and whether the performance hit is acceptable for now.

[j0sh]

@darkdarkdragon Correct, in the short term, we will probably end up being CPU constrained rather than memory constrained on broadcast nodes.

It'll be interesting to see exactly how much this affects our capacity on hardware that would otherwise be memory-limited. This shouldn't be a blocker for anything right now though.

FWIW, it takes roughly 100-120ms to decode a 8-second video on my i7 Macbook Pro using software decoding. Cloud GPUs should be in that ballpark. Resolving the LPMS bottleneck should help decoder performance as well.

For the future we have a few possible ways to mitigate the performance impact here:

• Running the pixel counting on another machine (perhaps as a pre-check alongside verification)
• Running the decode on a Nvidia GPU if the broadcaster has one available
• Adding VAAPI support to the broadcaster to allow decode acceleration on Intel QuickSync or AMD VCE. (Or perhaps videotoolbox for MacOS specifically.) This wouldn't help cloud broadcast instances (which typically don't have these type of hardware functions exposed) but may help for local testing, or testing on dedicated hardware.

[darkdarkdragon]

@j0sh as I understand, we need only number of frames? Can we parse video stream without decoding actual video data?

[j0sh]

@darkdarkdragon We would need at least the resolution of each frame as well, since each frame could potentially have a different resolution. Rather than return a long list of frames, we just return a sum of pixels for the segment. This certainly isn't foolproof from a verification perspective, but the goals here are mostly around reconciliation for PM accounting, rather than verification.

It is possible that we parse the bitstream and extract the necessary information needed to determine the number of pixels, without going through a full decode (in H.264 this is mostly the macroblock count, eg num_mbs_in_slice). FFmpeg has an internal API for this but it isn't exposed for library consumers, and I'm not keen on us writing custom bitstream parsing code at the moment. For the time being, this was the quickest way to get the job done.

[darkdarkdragon]

@j0sh thanks for the explanation! I didn't know that each frame could potentially have a different resolution.

FFmpeg has an internal API for this but it isn't exposed for library consumers,

Another reason to from FFmpeg, but certainly not now.