Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nvidia NVENC with non NV12 (or P010LE) pixel format corrupted video #35

Closed
bmegli opened this issue Jan 10, 2023 · 5 comments
Closed

Nvidia NVENC with non NV12 (or P010LE) pixel format corrupted video #35

bmegli opened this issue Jan 10, 2023 · 5 comments
Labels
bug Something isn't working

Comments

@bmegli
Copy link
Owner

bmegli commented Jan 10, 2023

May lead to corrupted data (here for BGR0 input):

image

@bmegli
Copy link
Owner Author

bmegli commented Jan 10, 2023

Quick Workaround

This is caused by changes in #26

It is enough to replace:

  • frames_ctx->sw_format = AV_PIX_FMT_NV12;
    if(hve_pixel_format_depth(h->sw_pix_fmt, &depth) != HVE_OK)
    return HVE_ERROR_MSG("failed to get pixel format depth");
    if(depth == 10)
    frames_ctx->sw_format = AV_PIX_FMT_P010LE;

With:

frames_ctx->sw_format = h->sw_pix_fmt;

As implemented before #26

@bmegli
Copy link
Owner Author

bmegli commented Jan 10, 2023

But we need a way to solve it for both VAAPI and NVENC at the same time

@bmegli bmegli added the bug Something isn't working label Jan 11, 2023
@bmegli
Copy link
Owner Author

bmegli commented Jan 11, 2023

But we need a way to solve it for both VAAPI and NVENC at the same time

Encoders

NVENC

ffmpeg -h encoder=h264_nvenc

Encoder h264_nvenc [NVIDIA NVENC H.264 encoder]:
    General capabilities: delay hardware 
    Threading capabilities: none
    Supported pixel formats: yuv420p nv12 p010le yuv444p p016le yuv444p16le bgr0 rgb0 cuda
# other options follow

Good - it can directly work from multiple pixel formats

VAAPI

ffmpeg -h encoder=h264_nvenc

Encoder h264_vaapi [H.264/AVC (VAAPI)]:
    General capabilities: delay 
    Threading capabilities: none
    Supported pixel formats: vaapi_vld

From #26 we know that:

  • in 3.x some format conversions happened "auto-magically"
  • in 4.x the same fail
    • on Intel it is possible to set hw frames sw format AV_PIX_FMT_NV12 or PO10LE and conversions will still happen
    • on AMD I have seen the same scenario fail

Full support might require:

  • explicit hw upload
  • with color conversion
  • but we would still need to specify to what we should convert

At least Intel will work to some extent with logic like in #26

@bmegli
Copy link
Owner Author

bmegli commented Jan 11, 2023

For now we will:

  • use frames_ctx->sw_format = h->sw_pix_fmt by default

Make special case for VAAPI to fallback to NV12 or P010LE depending on depth

I am not terribly happy but it will make Nvidia and Intel acceleration work in most scenarios

We will need to revisit it (VAAPI path):

  • when generalizing non NV12 (or P010LE) input between Intel and AMD
  • if using 4:2:2 or 4:4:4 chroma subsampling on modern hardware
  • if using bit depth beyond 10 bits on modern hardware
	frames_ctx->sw_format = h->sw_pix_fmt;
	
	// Starting from FFmpeg 4.1, avcodec will not fall back to NV12 automatically
	// when using non 4:2:0 software pixel format not supported by codec with VAAPI.
	// Here, instead of using h->sw_pix_fmt we always fall to P010LE for 10 bit
	// input and NV12 otherwise which may possibly lead to some loss of information
	// on modern hardware supporting 4:2:2 and 4:4:4 chroma subsampling
	// (e.g. HEVC with >= IceLake)
	// See:
	// https://github.com/bmegli/hardware-video-encoder/issues/26
	if(frames_ctx->format == AV_PIX_FMT_VAAPI)
	{
		frames_ctx->sw_format = AV_PIX_FMT_NV12;

		if(hve_pixel_format_depth(h->sw_pix_fmt, &depth) != HVE_OK)
			return HVE_ERROR_MSG("failed to get pixel format depth");

		if(depth == 10)
			frames_ctx->sw_format = AV_PIX_FMT_P010LE;
	}

bmegli added a commit that referenced this issue Jan 11, 2023
- make exception for VAAPI
- and try to use original sw_format otherwise

Related to: #35, #26
@bmegli
Copy link
Owner Author

bmegli commented Jan 11, 2023

Fixed

@bmegli bmegli closed this as completed Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant