Docs update rgb (#93)

* Remove period. Signed-off-by: [email protected] <[email protected]> * Remove the wedge_results folder. Signed-off-by: [email protected] <[email protected]> * Breaking out separate section for RGB encoding, removing it from QuickStart. Signed-off-by: [email protected] <[email protected]> * Initial commit of bit-depth testing, trying to determine how well the codec tracks Luminance. Signed-off-by: [email protected] <[email protected]> * Fleshing out the bit-depth comparison, giving a high level summary. Signed-off-by: [email protected] <[email protected]> * Adding some zscale links, and other formatting changes. Signed-off-by: [email protected] <[email protected]> * Fixing broken link. Signed-off-by: [email protected] <[email protected]> --------- Signed-off-by: [email protected] <[email protected]> Co-authored-by: [email protected] <[email protected]>
AcademySoftwareFoundation · May 15, 2024 · 1ea2b83 · 1ea2b83
1 parent 4f5941e
commit 1ea2b83
Show file tree

Hide file tree

Showing 10 changed files with 486 additions and 22 deletions.
diff --git a/.gitignore b/.gitignore
@@ -143,3 +143,4 @@ enctests/sources/enc_sources/chimera_cars_srgb_jpg
 enctests/sources/enc_sources/chimera_wind
 enctests/results.old
 tmp
+enctests/wedge_results
diff --git a/ColorPreservation.md b/ColorPreservation.md
@@ -13,7 +13,7 @@ We would like ffmpeg to do as little as possible in terms of color space convers
 
 The main problem is that ffmpeg by default assumes that any unknown still image format has a color space of [rec601](https://en.wikipedia.org/wiki/Rec._601) which is very unlikely to be the color space your source media was generate in. So unless you tell it otherwise it will attempt to convert from that colorspace producing a color shift.
 
-Separately, all the video formats typically do not use the full numeric range [0-255] but instead the Y' (luminance) channel have a nominal range of [16..235]  and the CB and CR channels have a nominal range of [16..240] with 128 as the neutral value. This frequently results in quantisation artifacts for 8-bit encoding (the standard for web playback). This fortunately is something you can change, [see TV vs. Full range](Quickstart.html#tv-vs-full-range-). below. The other option is to use higher bit depth, e.g. 10-bit or 12 bit for formats such as [ProRes](Encoding.html#prores-).
+Separately, all the video formats typically do not use the full numeric range [0-255] but instead the Y' (luminance) channel have a nominal range of [16..235]  and the CB and CR channels have a nominal range of [16..240] with 128 as the neutral value. This frequently results in quantization artifacts for 8-bit encoding (the standard for web playback). This fortunately is something you can change, [see TV vs. Full range](Quickstart.html#tv-vs-full-range-). below. The other option is to use higher bit depth, e.g. 10-bit or 12 bit for formats such as [ProRes](Encoding.html#prores-).
 
 Even if you are sticking to 8-bits encodes, if your source media is able to have a higher bit-depth (e.g. you are able to write out 16-bit PNG's to do the encode) it will help with the accuracy of the RGB to YUV conversion, particularly if you are using libswscale (see below).
 
@@ -27,7 +27,7 @@ For examples comparing these see: [here](https://academysoftwarefoundation.githu
 ```
 This is the most basic colorspace filtering. bt470bg is essentially part of the bt601 spec.  See: [https://www.ffmpeg.org/ffmpeg-filters.html#colormatrix](https://www.ffmpeg.org/ffmpeg-filters.html#colormatrix)
 
-e.g.
+Example:
 
 <!---
 name: test_colormatch_raw
@@ -63,7 +63,8 @@ There are a couple of issues with this filter:
  -vf "colorspace=bt709:iall=bt601-6-625:fast=1"
  ```
 Using colorspace filter, better quality filter, SIMD so faster too, can support 10-bit too.  The second part `-vf "colorspace=bt709:iall=bt601-6-625:fast=1"` encodes for the output being bt709, rather than the default bt601 matrix. iall=bt601-6-625 says to treat all the input (colorspace, primaries and transfer function) with the bt601-6-625 label). fast=1 skips gamma/primary conversion in a mathematically correct way.  See:  [https://ffmpeg.org/ffmpeg-filters.html#colorspace](https://ffmpeg.org/ffmpeg-filters.html#colorspace)
-e.g.
+
+Example:
 
 <!---
 name: test_colormatch_colorspace
@@ -91,11 +92,12 @@ ffmpeg -y -i ../sourceimages/chip-chart-1080-noicc.png \
 ```
 -vf "zscale=m=709:min=709:rangein=full:range=limited"
 ```
-Using the libswscale library. Seems similar to colorspace, but with image resizing, and levels built in.  [https://www.ffmpeg.org/ffmpeg-filters.html#scale-1](https://www.ffmpeg.org/ffmpeg-filters.html#scale-1)
 
-This is an alternative to libswscale, which does produce pretty good results for image resizing, but purely for RGB to YCrCb conversion libswscale appears very slightly better.
+[zscale](https://github.com/sekrit-twc/zimg) is an alternative to libswscale, which does produce pretty good results for image resizing, but purely for RGB to YCrCb conversion libswscale appears very slightly better. It is required to be explicitly compiled into ffmpeg.
 
-e.g.
+More detailed docs are: [here](http://underpop.online.fr/f/ffmpeg/help/zscale.htm.gz)
+
+Example:
 
 <!---
 name: test_colormatch_zscale
@@ -125,7 +127,8 @@ ffmpeg -y -i ../sourceimages/chip-chart-1080-noicc.png \
 Using the libswscale library. Seems similar to colorspace, but with image resizing, and levels built in.  [https://www.ffmpeg.org/ffmpeg-filters.html#scale-1](https://www.ffmpeg.org/ffmpeg-filters.html#scale-1)
 
 This is the recommended filter.
-e.g.
+
+Example:
 
 <!---
 name: test_colormatch_libswscale
@@ -141,7 +144,8 @@ comparisontest:
 -->
 ```
 ffmpeg -y -i ../sourceimages/chip-chart-1080-noicc.png \
-   -pix_fmt yuv444p10le -vf "scale=in_range=full:in_color_matrix=bt709:out_range=tv:out_color_matrix=bt709" \
+   -pix_fmt yuv444p10le \
+   -vf "scale=in_range=full:in_color_matrix=bt709:out_range=tv:out_color_matrix=bt709" \
    -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -qscale:v 1 \
    -color_range tv -colorspace bt709 -color_primaries bt709 -color_trc iec61966-2-1  \
    ./chip-chart-yuvconvert/spline444out_color_matrix.mp4

diff --git a/OtherFfmpegArgs.md b/OtherFfmpegArgs.md
@@ -1,6 +1,6 @@
 ---
 layout: default
-title: Useful Ffmpeg Filters.
+title: Useful Ffmpeg Filters
 nav_order: 5.5
 parent: Encoding Overview
 ---

diff --git a/Quickstart.md b/Quickstart.md
@@ -173,17 +173,6 @@ For more details see:
    * [Encoding Guide](Encoding.html#range)
 
 
-# Encoding as RGB. <a name="rgbencode"></a>
-You do not *have* to encode into YCrCb, h264 does support RGB encoding, which may be preferable in some situations.
-
-Using the encoder:
-```
--c:v libx264rgb
-```
-Will skip the conversion completely. Sadly this has no support in web browsers, but is supported by some players (e.g. RV). It is also limited to 8-bit.
-
-TODO Check about 10-bit encoding.
-
 For more details see:
    * [Comparing full-range vs. tv range](https://academysoftwarefoundation.github.io/EncodingGuidelines/tests/greyramp-fulltv/compare.html)
    * [Encoding Guide](Encoding.html#range)
diff --git a/RGBEncoding.md b/RGBEncoding.md
@@ -0,0 +1,86 @@
+---
+layout: default
+nav_order: 6
+title: RGB Encoding.
+parent: Encoding Overview
+---
+
+# RGB Encoding <a name="rgbencode"></a>
+You do not *have* to encode into YCrCb, there are a number of codecs that support RGB directly.
+
+It does lower the compression efficiency a little, but eliminates any concern that the YCrCb conversion is affecting the imagery.
+
+| Codec Family | ffmpeg codec | Bit depth |  pix_fmt | Web Support | 
+|------------|------------|------------|------------|
+| [h264](Encodeh264.html) | libx264rgb | 8 |  | no | 
+| [HEVC/H265](EncodeHevc.html) | libx265 | 8 10 12 | rgb rgb-10 rgb-12 | All browsers | 
+| [HEVC/H265](EncodeHevc.html) | hevc_videotoolbox | 8 10 | rgb rgb-10 rgb-12 | All browsers | 
+| [VP9](EncodeVP9.html) | libvpx-vp9 | 8 10 12 | rgb rgb-10 rgb-12 | All Browsers | 
+| [Mjpeg](EncodeMJPEG.html) | mjpeg | 8 | yuv-4:2:2 yuv-4:4:4 | no | 
+| [DNxHD](EncodeDNxHD.html) | dnxhd | 8 10 | rgb | no | 
+
+
+## H264 RGB Encoding
+
+h264 does support RGB encoding, which may be preferable in some situations.
+
+Using the encoder:
+```
+-c:v libx264rgb
+```
+
+Has no support in web browsers, but there is limited support in players such as RV.
+
+## mjpeg RGB Encoding
+
+For more details see: [Mjpeg](EncodeMJPEG.html). This is an 8-bit only encode.
+
+```
+	-c:v mjpeg -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb -pix_fmt rgb24 -q:v 2 -vf "scale=in_range=full:out_range=full"
+```
+
+## VP9 RGB Encoding
+
+VP9 has some excellent 10 and 12 bit RGB encodes.
+
+Example encoding:
+
+<!---
+name: test_vp9
+sources: 
+- sourceimages/chip-chart-1080-16bit-noicc.png.yml
+comparisontest:
+   - testtype: idiff
+   - testtype: assertresults
+     tests:
+     - assert: less
+       value: max_error
+       less: 0.00195
+-->
+```
+ffmpeg -r 24 -start_number 1 -i inputfile.%04d.png -frames:v 200 -c:v libvpx-vp9 \
+    -crf 22 -pix_fmt gbrp10le -quality good -row-mt 1 -speed 2 -vf "scale=in_range=full:out_range=full" \
+    -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb   \
+     -y outputfile.mp4
+```
+
+
+## RGB Comparisons
+
+
+| *label* | *Codec Params* |
+| 8_bit_libx264rgb | -c:v libx264rgb -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb -crf 18 -pix_fmt rgb24 -preset slow -vf "scale=in_range=full:out_range=full" -x264-params keyint=15:no-deblock=1 |
+| 8bit_mjpeg | -c:v mjpeg -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb -pix_fmt rgb24 -q:v 2 -vf "scale=in_range=full:out_range=full" |
+| vp9_10bit_rgb | -c:v libvpx-vp9 -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb -crf 22 -pix_fmt gbrp10le -quality good -row-mt 1 -speed 2 -vf "scale=in_range=full:out_range=full" |
+| vp9_12bit_rgb | -c:v libvpx-vp9 -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb -crf 22 -pix_fmt gbrp12le -quality good -row-mt 1 -speed 2 -vf "scale=in_range=full:out_range=full" |
+| vp9_8bit_rgb | -c:v libvpx-vp9 -color_primaries bt709 -color_range pc -color_trc bt709 -colorspace rgb -crf 22 -pix_fmt gbrp -quality good -vf "scale=in_range=full:out_range |
+
+| ![](enctests/reference-results/rgb-tests-filesize.png) This is showing different rgb encoding against file size. |
+| ![](enctests/reference-results/rgb-tests-encode_time.png) This is showing different rgb encoding against encoding time |
+| ![](enctests/reference-results/rgb-tests-vmaf_harmonic_mean.png) This is showing different rgb encoding against mean VMAF |
+
+
+
+For more details see:
+   * [Comparing full-range vs. tv range](https://academysoftwarefoundation.github.io/EncodingGuidelines/tests/greyramp-fulltv/compare.html)
+   * [Encoding Guide](Encoding.html#range)
diff --git a/enctests/bitDepth/BitDepthComparisons.md b/enctests/bitDepth/BitDepthComparisons.md
@@ -0,0 +1,91 @@
+---
+layout: default
+nav_order: 6.08
+title: Bit Depth Analysis
+parent: Test Output
+---
+
+# Bit Depth Analysis
+
+Sometimes its useful to know what the actual dynamic range of a particular combination of codec, profile and pixel format is, since it may not necessarily be what you think. i.e. you think you are getting 10-bits (i.e. 1024 values) when in fact you could be getting quite a bit less.
+
+This work is inspired by [https://github.com/ColorlabMD/Prores-BitDepth](https://github.com/ColorlabMD/Prores-BitDepth).
+
+These tests generate a luminance YCrCb flat image, where each frame increments the luminance by one using the geq ffmpeg filter.
+
+For example for the prores_ks codec we would run:
+
+```
+ffmpeg -y -r 24 -f lavfi -i nullsrc=s=720x480,format=yuv444p10le -frames:v 1024 -vf geq=N:512:512 -c:v prores_ks -profile:v 4444xq ./colors-prores_ks_10_4444xq.mov
+```
+
+We then read the resulting movie file, to see if the luminance that we expect.
+
+The columns are defined as:
+   * Test Name - which codec we are testing.
+   * Bit Depth - What bit-depth the codec is being evaluated to (which determines how many unique values to test to).
+   * Unique values - How many resulting values are unique, but may not be exactly the same value.
+   * STDDEV > 0.0001 - Standard deviation of the frame is > 0.0001 which means what should be a flat color, isnt.
+   * Off by 1 - The color is a flat color, but is "off by 1" (i.e. what should be 3 is 4), again not ideal.
+   * Other invalid - is a flat color, but is some other value than expected.
+
+## Running the tests
+
+To run the tests you need three additional python libraries as well as ffmpeg:
+
+```
+pip install yuvio numpy pyseq 
+```
+
+Warning: The file sizes can get large, so expect it to generate 8GB of data.
+
+Note, the one thing the ColorlabMD test does that this does not currently do is to also do a comparison of chroma variation.
+
+## Results
+
+### Prores_ks
+
+Tests - prores_ks_10_4444xq, prores_ks_10_proxy, prores_ks_10_hq
+
+It shows that the ffmpeg_ks encoder is only encoding to the legal 10-bit range (i.e. 877 values), which is pretty much what we expect.
+
+The tests [https://github.com/ColorlabMD/Prores-BitDepth](https://github.com/ColorlabMD/Prores-BitDepth) tests seem a little unfair on prores_ks, which clearly defines itself as a 10-bit codec, and I suspect that the apple encoding is setting full range values which Prores (in this case) is not.
+
+Note, Prores_ks will read 12 bit files, just not generate them.
+
+### Apple Videotoolbox Prores
+
+Tests - prores_videotoolbox_10_proxy, prores_videotoolbox_10_hq, prores_videotoolbox_10_4444, prores_videotoolbox_12_xq
+
+Interestingly, we are getting better results for prores_ks for proxy and HQ, in that the values are at least consistent, which for videotoolbox they are often off by 1.
+
+However, for 4444 and XQ they are good.
+
+### DNxHD
+
+All the values are as expected.
+
+### h264
+
+All the values are as expected.
+
+### x265 HEVC
+
+x265 HEVC is struggling with flat color with quite a few values not generating a uniform value.
+
+### VP9 
+
+All the values are as expected.
+
+### libsvtav1
+
+Something rather odd going on here, the 10-bit encode is generating only 475 unique values rather than 1024. Looking at the values, it seems like there is some sort of rounding going on here.
+
+### libaom-av1
+
+Is slightly better, with 866 unique values, but many of them are off by one or more. At 12-bit the results are similar.
+
+
+Click [here](bitDepthResults.html) to see the full page table.
+
+{% include_relative bitDepthResults.html %}