Bandwidth utilisation not adjustable

[Changliu52]

Hi @bmegli

I was testing the performance of the encoding with different bandwidth parameters

the initilisation I used was:

hw_config_.pixel_format = "nv12";
hw_config_.profile = FF_PROFILE_H264_CONSTRAINED_BASELINE;
hw_config_.encoder = NULL;
hw_config_.width = 1280;
hw_config_.height= 720;
hw_config_.framerate = 30;
hw_config_.max_b_frames = 0;
hw_config_.device = NULL; //NULL as last argv argument, or device path
hw_config_.bit_rate = 4000000;

However, the traffic monitoring in linux only showing 1.8-2 mbps. I have checked different bit_rate parameters (10,000,000 and 20,000,000) but not really made any difference. Did I reached some limitations of the hardware? (not sure if the h264 is able to handle 1280x720 resolution.

Also I did have two NHVE initialised with different bit rate and different port. would that make a difference?

Best Regards,
Chang

[bmegli]

I think I have seen such problem before with a combination of driver, libva and ffmpeg version, for example here

19.04
I used it on m3-8100Y with GPD Win2, worked out of the box with:
Ubuntu MATE 19.04 for GPD Pocket 2 & GPD WIN 2
H264 and HEVC Main10 worked however I had doubts whether bitrate control worked correctly. Didn't seem so.

When live streaming complex material it should be easy to spot, the quality is simply bad.

What platform are you on (generation), which driver do you use, what OS/FFmpeg version?

I would start debugging it by:

• enabling verbose/debug FFmpeg logging options in HVE
• looking what information about VAAPI rate control is emmitted
• try to switch driver if it is possible (i965 vs iHD) for your generation
• try to switch to CBR instead of VBR (the default) using command line FFmpeg and see if it helps like here in "Mapping options from libx264" (I am not sure it is entirely up to date with new FFMpeg versions)

Also I did have two NHVE initialised with different bit rate and different port. would that make a difference?

Should not in theory but (as side note) I would check gpu usage with:

sudo intel_gpu_top

H.264 should not require much really but HEVC depending on platform generation can be heavy on GPU, see for example this where double HEVC stream uses 65% of SBCs gpu and barely keeps up to 30 FPS. Just in case, you may control speed/quality trade-off with compression_level.

[bmegli]

Also if your material is not complex (e.g. static scene) from what I remember you may get low bitrate with VBR.

Make sure you have complex scene (e.g. moving camera) that actually requires some bitrate.

[Changliu52]

Thanks @bmegli
I'm running ubuntu 18.04 on UPS-EDGE-X70864-U01.
Will test the suggestions and report back.

Best Regards

[Changliu52]

Intel ATOM Quad-Core up to 2.0GHz x7-E3950
Graphic: Intel HD Graphic 500

[bmegli]

Looking at ark it is ApolloLake

By cross-checking with FFmpeg QuickSync it has gen 9 graphics.

Apollo Lake	gen9	VP9, H.265 Main10 decode.

By looking at table Intel open source media stack in link above we see:

Project Name	Supported Gen Graphics	Open Source Repo
MSDK	gen8+	​https://github.com/Intel-Media-SDK/MediaSDK
Libva	gen5+	​https://github.com/intel/libva
i965 driver	gen5 ~ gen9.5	​https://github.com/intel/intel-vaapi-driver
iHD driver	gen8+	​https://github.com/intel/media-driver

that it should probably work with both i965 and iHD.

Crosschecking with iHD support matrix we see that even with iHD only it has two different ways to encode H.264

E - Hardware Encoding, Low Power Encoding(VDEnc/Huc)
Es - Hardware(PAK) + Shader(media kernel+VME) Encoding

Es only with full featured build.

To get full feature build in Ubuntu

# this is for iHD full feature build
sudo apt install intel-media-va-driver-non-free

Ideally also before and after updating to see if it changes anything:

LIBVA_DRIVER_NAME=iHD vainfo --display drm --device /dev/dri/renderD128

[bmegli]

Quoting myself:

To get full feature build in Ubuntu

# this is for iHD full feature build
sudo apt install intel-media-va-driver-non-free

Ideally also before and after updating to see if it changes anything:

LIBVA_DRIVER_NAME=iHD vainfo --display drm --device /dev/dri/renderD128

Sorry - the above was for Ubuntu 20.04 (which has both i965 and iHD drivers), not 18.04 (which has i965 driver)

[bmegli]

@Changliu52

You can also:

1. Check with vainfo if your hardware supports alternative encoding paths:

vainfo --display drm --device /dev/dri/renderD128
#VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
# alternative Low Power fixed function encoding
#VAProfileH264ConstrainedBaseline:	VAEntrypointEncSliceLP

2. Check issues (and solutions) for the vaapi driver

(search for bitrate, rate control, etc, both in open and closed)

3. Try Ubuntu 20.04 which has newer FFmpeg and both i965 and iHD drivers

Some of the issues with 20.04 are already fixed

[Changliu52]

Thank you so much for the suggestions @bmegli
I have tried sudo apt install intel-media-va-driver-non-free and did some search around.
actually ubuntu 18.04 doesn't yet support that.

I did some further test and realised that the intel-gpu-top gives about 10% utilisation, and i think because of some other part of the code, the CPU is pretty busy like 80% in all cores. and checked my raw depth images is about 15 FPS instead of 30 FPS as I set.

Not sure why this happened, but when I set the framerate to 15 in for NHVE, the bitrate control seems working.
I can now set the bitrate and the network monitor responds, although numbers is normally about half of the bitrate I set. I guess the bitrate setting is the maximum number while the network monitor shows the averaged activities in the network.

[bmegli]

@Changliu52

The default rate control is VBR

If you want CBR replace HVE line

with:

h->avctx->bit_rate = h->avctx->rc_max_rate = config->bit_rate;

This will probably keep you closer to target bitrate.