HDR PQ and Source updates (#31)

* Adding PQ Encoding. Signed-off-by: [email protected] <[email protected]> * Script for copying over the data used in the documentation from the tests. Signed-off-by: [email protected] <[email protected]> * Updating the YML file spec, so that its relative. This means we can check the files in, and it should work when we check it out. Signed-off-by: [email protected] <[email protected]> * Fixing readme, since this is now just SDR content. Signed-off-by: [email protected] <[email protected]> * Adding callout. Signed-off-by: [email protected] <[email protected]> --------- Signed-off-by: [email protected] <[email protected]> Co-authored-by: [email protected] <[email protected]>
AcademySoftwareFoundation · Nov 6, 2023 · f6051f7 · f6051f7
1 parent 6e46690
commit f6051f7
Show file tree

Hide file tree

Showing 7 changed files with 150 additions and 22 deletions.
diff --git a/enctests/HDR_Encoding.md b/enctests/HDR_Encoding.md
@@ -7,16 +7,98 @@ parent: Encoding Overview
 
 # HDR Encoding.
 
-*This is under development*
+{: .warning } This is currently under development, use at your own risk.
 
+There are a lot of competing standards for [HDR](https://www.lightillusion.com/what_is_hdr.html), we are going to focus only on two due to their relative simplicity (not requiring dynamic metadata), and lack of licensing fees. 
+1. [HLG](https://en.wikipedia.org/wiki/Hybrid_log%E2%80%93gamma) encoding, developed by NHK and the BBC.
+2. [PQ10](https://en.wikipedia.org/wiki/HDR10) which uses the [PQ EOTF](https://en.wikipedia.org/wiki/Perceptual_quantizer). 
 
-## Frame prep
+## PQ10
 
-There are a two main encoding formats [HLG](https://en.wikipedia.org/wiki/Hybrid_log%E2%80%93gamma) and [PQ](https://en.wikipedia.org/wiki/Perceptual_quantizer) encoding. We are choosing HLG since its a slightly simpler format, and requires less additional metadata.
+Is the HDR format which is the HDR10 without the metadata. It uses the [PQ EOTF](https://en.wikipedia.org/wiki/Perceptual_quantizer) aka SMPTE ST 2084 otherwise is somewhat similar to create as HLG, but there may be different reasons for choosing each.
+
+### Frame prep
 
 We take advantage of ACES to do the initial conversion to an intermediate format, which we are using png as the container.
 
-```consoleconsole
+```console
+oiiotool -v --framepadding 5 --frames 6700-6899 sparks2/SPARKS_ACES_#.exr --resize 1920x1014 \
+      --colorconvert acescg out_rec2020st20842000nits -d uint16 -o sparks2_pq2000/sparks2_pq2000.#.png
+```
+
+| --- | --- |
+|--colorconvert acescg out_rec2020st20842000nits | This is the core colorspace conversion, out_rec2020st20842000nits is an output colorspace conversion for rec2020 PQ at 2000 nit display |
+| -d uint16 | Output as 16-bit file format |
+
+
+
+### PQ 444 FFMPEG encoding
+
+```console
+ffmpeg  -sws_flags print_info+accurate_rnd+bitexact+full_chroma_int   \
+    -color_range pc   -color_trc smpte2084   -color_primaries bt2020   -colorspace bt2020nc   \
+    -pix_fmt rgb48be  -r 30 -start_number 6700 -i sparks2_pq2000/sparks2_pq2000.%05d.png   \
+    -c:v libx265   \
+	-tag:v hvc1  \
+    -color_range tv   -color_trc smpte2084   -color_primaries bt2020   -colorspace bt2020nc  \
+    -pix_fmt yuv444p10le   -sws_flags print_info+accurate_rnd+bitexact+full_chroma_int  \
+    -x265-params 'colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:range=limited:master-display=G\(13250,34500\)B\(7500,3000\)R\(34000,16000\)WP\(15635,16450\)L\(10000000,1\):max-cll=2000,400'   \
+    sparks2_pq2000_444.mov
+```
+
+NOTE, this is a little different to other conversions (is this better?). We are defining up front what the source media is defined (e.g. `-color_range pc   -color_trc smpte2084   -color_primaries bt2020   -colorspace bt2020nc` ). 
+
+#### Source media definition.
+
+| --- | --- |
+| -color_range pc | Set the source range to be full-range |
+| -color_trc smpte2084 | smpte2084 is the PQ reference EOTF |
+| -color_primaries bt2020 | Use the bt2020 color primaries |
+| -colorspace bt2020nc | Tagging the YcBCr as being encoded using the BT-2020 non-constant luminance. |
+| -pix_fmt rgb48be | We are assuming 16-bit RGB imagery as input |
+
+
+#### Overall encode params
+
+| --- | --- |
+| -c:v libx265 | Use the h265 encoder |
+| -tag:v hvc1 | Tag the file for playback on mac | 
+
+#### Encode media definition
+
+| --- | --- |
+| -color_range tv | Set the source range to be tv range. |
+| -color_trc smpte2084 | smpte2084 is the PQ reference EOTF |
+| -color_primaries bt2020 | Use the bt2020 color primaries |
+| -colorspace bt2020nc | Tagging the YcBCr as being encoded using the BT-2020 non-constant luminance. |
+| -pix_fmt yuv444p10le | YUV 444 10-bit output
+
+#### X265 parameters
+
+We explicitly define the X265 parameters (see [x265](https://x265.readthedocs.io/en/2.5/cli.html) )
+
+| --- | --- |
+| colorprim=bt2020 | Set the colorprimaries to bt2020 |
+| transfer=smpte2084 | Set the ETOF to PQ (aka. smpte2084 ) |
+| colormatrix=bt2020nc | UTagging the YcBCr as being encoded using the BT-2020 non-constant luminance. |
+| range=limited | Set the source range to be tv range. |
+| master-display=G\(13250,34500\)B\(7500,3000\)R\(34000,16000\)WP\(15635,16450\)L\(10000000,1\) | SMPTE ST 2086 mastering display color volume SEI info, specified as a string which is parsed when the stream header Essentially setting the X,Y display primaries for rec2020 along with the Whitepoint, and the Max,min luminance values in units of 0.00001 NITs. See the above docs for more info. |
+| max-cll=2000,400 | Set the Maximum content light level (in this case 2000 nits = max content light level and 400 = the MaxFall - the maxiumum frame-average light level) |
+
+## HLG
+
+[HLG](https://en.wikipedia.org/wiki/Hybrid_log%E2%80%93gamma) encoding, developed by NHK and the BBC. Its designed to be backwards compatible with the EOTF to SDR but with an exension for higher intensities. It has a more limited maximum peak brightness of 5000 nits rather than 10000 nits for PQ. 
+
+It is also designed to adapt to the surrounding room light levels, unlike PQ which is fixed. The downside is that this adaption could affect the media in unexpected ways.
+
+
+### Frame prep
+
+
+
+We take advantage of ACES to do the initial conversion to an intermediate format, which we are using png as the container.
+
+```console
 oiiotool -v --framepadding 5 --frames 6700-6899 sparks2/SPARKS_ACES_#.exr --resize 1920x1014 \
       --colorconvert acescg out_rec2020hlg1000nits -d uint16 -o sparks2_hlg/sparks2_hlg.#.png
 ```
@@ -27,7 +109,7 @@ oiiotool -v --framepadding 5 --frames 6700-6899 sparks2/SPARKS_ACES_#.exr --resi
 
 
 
-## HLG 444 FFMPEG encoding
+### HLG 444 FFMPEG encoding
 
 ```console
 ffmpeg  -sws_flags print_info+accurate_rnd+bitexact+full_chroma_int   \
@@ -43,7 +125,7 @@ ffmpeg  -sws_flags print_info+accurate_rnd+bitexact+full_chroma_int   \
 
 NOTE, this is a little different to other conversions (is this better?). We are defining up front what the source media is defined (e.g. `-color_range pc   -color_trc arib-std-b67   -color_primaries bt2020   -colorspace bt2020nc` ). 
 
-### Source media definition.
+#### Source media definition.
 
 | --- | --- |
 | -color_range pc | Set the source range to be full-range |
@@ -53,13 +135,13 @@ NOTE, this is a little different to other conversions (is this better?). We are
 | -pix_fmt rgb48be | We are assuming 16-bit RGB imagery as input |
 
 
-### Overall encode params
+#### Overall encode params
 
 | --- | --- |
 | -c:v libx265 | Use the h265 encoder |
 | -tag:v hvc1 | Tag the file for playback on mac | 
 
-### Encode media definition
+#### Encode media definition
 
 | --- | --- |
 | -color_range tv | Set the source range to be tv range. |
@@ -68,7 +150,7 @@ NOTE, this is a little different to other conversions (is this better?). We are
 | -colorspace bt2020nc | Tagging the YcBCr as being encoded using the BT-2020 non-constant luminance. |
 | -pix_fmt yuv444p10le | YUV 444 10-bit output
 
-### X265 parameters
+#### X265 parameters
 
 We explicitly define the X265 parameters (see [x265](https://x265.readthedocs.io/en/2.5/cli.html) )
 
@@ -78,10 +160,12 @@ We explicitly define the X265 parameters (see [x265](https://x265.readthedocs.io
 | colormatrix=bt2020nc | UTagging the YcBCr as being encoded using the BT-2020 non-constant luminance. |
 | range=limited | Set the source range to be tv range. |
 | master-display=G\(13250,34500\)B\(7500,3000\)R\(34000,16000\)WP\(15635,16450\)L\(10000000,1\) | SMPTE ST 2086 mastering display color volume SEI info, specified as a string which is parsed when the stream header Essentially setting the X,Y display primaries for rec2020 along with the Whitepoint, and the Max,min luminance values in units of 0.00001 NITs. See the above docs for more info. |
-| max-cll=1000,400 | Set the Maximum content light level |
+| max-cll=1000,400 | Set the Maximum content light level (in this case 2000 nits = max content light level and 400 = the MaxFall - the maxiumum frame-average light level) |
+
+TODO - We do need to understand if the max-cll and master-display parameters are even used for HLG display.
 
 
-## HLG 420 FFMPEG encoding
+### HLG 420 FFMPEG encoding
 
 ```console
 ffmpeg  -sws_flags print_info+accurate_rnd+bitexact+full_chroma_int   \
@@ -98,7 +182,7 @@ ffmpeg  -sws_flags print_info+accurate_rnd+bitexact+full_chroma_int   \
 ```
 
 
-## HLG 422 FFMPEG Encoding
+### HLG 422 FFMPEG Encoding
 
 ```console
 ffmpeg  -sws_flags print_info+accurate_rnd+bitexact+full_chroma_int   \
@@ -115,3 +199,4 @@ ffmpeg  -sws_flags print_info+accurate_rnd+bitexact+full_chroma_int   \
 
 Further Reading:
    * [H.273 Specification.](https://www.itu.int/rec/T-REC-H.273-202107-I/en)
+   * https://www.lightillusion.com/what_is_hdr.html
diff --git a/enctests/reference-results/copy-results.sh b/enctests/reference-results/copy-results.sh
@@ -0,0 +1,42 @@
+cp ../h264-crf-results.otio .
+cp ../h264-crf-encode/h264-crf-test-encode_time.png .
+cp ../h264-crf-encode/h264-crf-test-filesize.png .
+cp ../h264-crf-encode/h264-crf-test-vmaf_harmonic_mean.png .
+cp ../h264-results.otio .
+cp ../h264-encode/h264-test-encode_time.png .
+cp ../h264-encode/h264-test-filesize.png .
+cp ../h264-encode/h264-test-vmaf_harmonic_mean.png .
+cp ../vp9-crf2-results.otio .
+cp ../vp9-crf2-encode/vp9-crf2-test-encode_time.png .
+cp ../vp9-crf2-encode/vp9-crf2-test-filesize.png .
+cp ../vp9-crf2-encode/vp9-crf2-test-vmaf_harmonic_mean.png .
+cp ../prores-results.otio .
+cp ../prores-encode/prores-test-encode_time.png .
+cp ../prores-encode/prores-test-filesize.png .
+cp ../prores-encode/prores-test-vmaf_harmonic_mean.png .
+cp ../prores-qscale-results.otio .
+cp ../prores-qscale-encode/prores-qscale-tests-encode_time.png .
+cp ../prores-qscale-encode/prores-qscale-tests-filesize.png .
+cp ../prores-qscale-encode/prores-qscale-tests-vmaf_harmonic_mean.png .
+cp ../av1-crf-results.otio .
+cp ../av1-crf-encode/av1-crf-test-encode_time.png .
+cp ../av1-crf-encode/av1-crf-test-filesize.png .
+cp ../av1-crf-encode/av1-crf-test-vmaf_harmonic_mean.png .
+cp ../hevc-crf-results.otio .
+cp ../hevc-crf-encode/hevc-crf-test-encode_time.png .
+cp ../hevc-crf-encode/hevc-crf-test-filesize.png .
+cp ../hevc-crf-encode/hevc-crf-test-vmaf_harmonic_mean.png .
+cp ../hevc-results.otio .
+cp ../hevc-encode/hevc-test-encode_time.png .
+cp ../hevc-encode/hevc-test-filesize.png .
+cp ../hevc-encode/hevc-test-vmaf_harmonic_mean.png .
+cp ../codec-results.otio .
+cp ../codec-encode/codec-test-encode_time.png .
+cp ../codec-encode/codec-test-filesize.png .
+cp ../codec-encode/codec-test-vmaf_harmonic_mean.png .
+cp ../codec-encode/codec-test-psnr_y_harmonic_mean.png .
+cp ../mjpeg-qscale-results.otio .
+cp ../mjpeg-qscale-encode/mjpeg-qscale-tests-encode_time.png .
+cp ../mjpeg-qscale-encode/mjpeg-qscale-tests-filesize.png .
+cp ../mjpeg-qscale-encode/mjpeg-qscale-tests-vmaf_harmonic_mean.png .
+cp ../mjpeg-qscale-encode/mjpeg-qscale-tests-psnr_y_harmonic_mean.png .
diff --git a/enctests/sources/enc_sources/README.md b/enctests/sources/enc_sources/README.md
@@ -2,10 +2,10 @@
 layout: default
 nav_order: 7.1
 title: Reference Source Test Media
-parent: ORI Video Encoding Test Framework
+parent: ORI Encoding Test Framework
 ---
 
-# Reference Source Test Media for HDR Media.
+# Reference Source Test Media for Encoding
 
 We have collected a number of source clips that you can use to test on, these are mostly from the excellent netflix [Open Source Content](https://opencontent.netflix.com/) site. Some of the clips are quite large, so we have tried to extract shorter 100-200 frame clips for the encoding tests, and also extracting individual clips from edited sequences.
 
@@ -19,6 +19,7 @@ The clips that are created include:
 
 
 Other media that could be useful include:
+
 | ![sintel trailer](thumbnails/sintel_trailer_2k_0591.jpg) | [sintel 1080p-png trailer download](https://s3.amazonaws.com/senkorasic.com/test-media/video/sintel/source/Sintel-trailer-1080p-png.zip) | Blender CG created media. |
 
 

diff --git a/enctests/sources/enc_sources/chimera_cars_srgb/chimera_cars_srgb.%05d.png.yml b/enctests/sources/enc_sources/chimera_cars_srgb/chimera_cars_srgb.%05d.png.yml
@@ -1,6 +1,6 @@
 images: true
-path: /Users/sam/git/EncodingGuidelines/enctests/sources/enc_sources/chimera_cars_srgb/chimera_cars_srgb.%05d.png
-width: 2048
+path: chimera_cars_srgb.%05d.png
+width: 1920
 height: 1080
 pix_fmt: rgb48be
 in: 2500

diff --git a/enctests/sources/enc_sources/chimera_coaster_srgb/chimera_coaster_srgb.%06d.png.yml b/enctests/sources/enc_sources/chimera_coaster_srgb/chimera_coaster_srgb.%06d.png.yml
@@ -1,6 +1,6 @@
 images: true
-path: /Users/sam/git/EncodingGuidelines/enctests/sources/enc_sources/chimera_coaster_srgb/chimera_coaster_srgb.%06d.png
-width: 2048
+path: chimera_coaster_srgb.%06d.png
+width: 1920
 height: 1080
 pix_fmt: rgb48be
 in: 44200

diff --git a/enctests/sources/enc_sources/chimera_fountains_srgb/chimera_fountains_srgb.%05d.png.yml b/enctests/sources/enc_sources/chimera_fountains_srgb/chimera_fountains_srgb.%05d.png.yml
@@ -1,6 +1,6 @@
 images: true
-path: /Users/sam/git/EncodingGuidelines/enctests/sources/enc_sources/chimera_fountains_srgb/chimera_fountains_srgb.%05d.png
-width: 2048
+path: chimera_fountains_srgb.%05d.png
+width: 1920
 height: 1080
 pix_fmt: rgb48be
 in: 5400

diff --git a/enctests/sources/hdr_sources/sparks_srgb/sparks_srgb.%05d.png.yml b/enctests/sources/hdr_sources/sparks_srgb/sparks_srgb.%05d.png.yml
@@ -1,8 +1,8 @@
 images: true
-path: /Users/sam/git/EncodingGuidelines/enctests/sources/hdr_sources/sparks_srgb/sparks_srgb.%05d.png
+path: sparks_srgb.%05d.png
 width: 1920
 height: 1014
 pix_fmt: rgb24
 in: 6100
 duration: 200
-rate: 25.0
+rate: 24