Skip to content

Commit 318e4c2

Browse files
mournermourner
committed
switch OKLab HyAB perceptual color difference
1 parent 3f47d18 commit 318e4c2

13 files changed

Lines changed: 304 additions & 36 deletions

File tree

‎PLAN.md‎

Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
# Plan: replace YIQ with OKLab as the perceptual color difference metric
2+
3+
## Motivation
4+
5+
The current metric (Kotsarenko/Ramos YIQ Euclidean distance) operates on gamma-encoded sRGB
6+
with a purely linear transform, so it structurally cannot account for the nonlinearity of
7+
human color perception — no re-tuning of the Y/I/Q weights can fix it. Concrete failures
8+
from [#127](https://github.com/mapbox/pixelmatch/issues/127), with distances normalized so
9+
black↔white = 1.0 (default threshold is 0.1):
10+
11+
| case | YIQ (current) | OKLab | OKLab HyAB | CIEDE2000 | human verdict |
12+
|---|---|---|---|---|---|
13+
| rgb(41,56,157) vs rgb(41,56,0) | 0.199 | 0.247 | 0.315 | 0.582 | very different |
14+
| rgb(0,254,252) vs rgb(94,254,252) | **0.200** | 0.025 | 0.034 | 0.024 | nearly identical |
15+
| rgb(39,44,92) vs rgb(39,44,14) | 0.099 | 0.136 | 0.167 | 0.391 | clearly different |
16+
| rgb(0,254,252) vs rgb(47,254,252) | 0.100 | 0.007 | 0.009 | 0.006 | imperceptible |
17+
18+
YIQ assigns the same distance (0.199 vs 0.200) to a pair that is "very different" and a pair
19+
that is "nearly identical". OKLab separates them 10x and agrees with CIEDE2000 on every ranking.
20+
21+
## Why OKLab (and not CIEDE2000, redmean, ICtCp)
22+
23+
- **OKLab** (Ottosson, 2020; adopted by CSS Color 4) was explicitly fitted so that plain
24+
Euclidean distance approximates CIEDE2000 predictions. The transform is cheap:
25+
sRGB→linear (256-entry LUT) → 3×3 matrix → cbrt per channel → 3×3 matrix.
26+
It keeps pixelmatch's squared-distance trick (no sqrt; black↔white squared distance is
27+
exactly 1.0) and the brighter/darker sign convention falls out of ΔL.
28+
- **CIEDE2000** is ~60x slower per pair (~2.5–4x end-to-end), and was fitted on *small*
29+
near-JND differences (textile/patch studies) — a regime pixelmatch doesn't operate in;
30+
it's known to be non-uniform for large differences. Not worth it.
31+
- **redmean** is cheap but shares YIQ's structural flaw (scores the "nearly identical"
32+
cyan pair at 0.182 — nearly as bad as YIQ).
33+
- **ICtCp/ΔEITP** is comparable to OKLab in cost/quality but designed for HDR; no advantage
34+
for 8-bit sRGB.
35+
36+
## Measured performance and diff counts
37+
38+
Per-pair (random colors): YIQ 250M pairs/s; naive OKLab (`Math.cbrt`) 22M; OKLab with a
39+
4096-entry interpolated cbrt LUT 37M (max distance error 2.6e-4 — negligible vs any
40+
threshold). Per-pair cost is heavily amortized end-to-end because `colorDelta` only runs for
41+
differing pixels; the identical-pixel u32 scan and AA detection dominate. Memoizing
42+
conversions per unique color does not help (JS Map overhead ≈ the compute itself).
43+
44+
End-to-end on `bench.js` (real fixtures, 100 iterations), plus per-fixture diff counts at
45+
threshold 0.1:
46+
47+
threshold 0.1 (timings from isolated node processes, best of 5 — mixing variants in one
48+
process pollutes the JIT and skews results by ~10%):
49+
50+
| variant | bench.js | fx1 | fx2 | fx3 | fx4 | fx5 | fx6 | fx7 |
51+
|---|---|---|---|---|---|---|---|---|
52+
| YIQ (current) | 799 ms | 106 | 9730 | 178 | 6889 | 6 | 51 | 2448 |
53+
| OKLab Euclidean | 950 ms (**+19%**) | 111 | 9148 | 164 | 4760 | 6 | 38 | 2330 |
54+
| OKLab HyAB | 1060 ms (**+33%**) | 116 | 10155 | 187 | 6942 | 6 | 38 | 2348 |
55+
| OKLab HyAB, sqrt-free | 991 ms (**+24%**) | (identical to HyAB) | | | | | | |
56+
57+
### HyAB optimization: sqrt-free thresholding — measured
58+
59+
`colorDelta`'s return value is only ever used two ways: compared against `maxDelta`, and its
60+
sign checked for `diffColorAlt`. So the HyAB sqrt can be eliminated by moving the threshold
61+
test inside `colorDelta`: `|ΔL| + √(Δa²+Δb²) > t` ⇔ `|ΔL| > t`, or else
62+
`Δa²+Δb² > (t − |ΔL|)²`. Bonus: when `|ΔL|` alone exceeds the threshold, Δa/Δb aren't
63+
computed at all. `colorDelta(…, maxDelta)` then returns just `0 / +1 / −1` (sign = darker
64+
or brighter). Verified to produce bit-identical diff counts on all fixtures.
65+
66+
This recovers over half of HyAB's overhead: **+33% → +24%** vs YIQ (~4% over Euclidean).
67+
Also tested: folding the LMS matrix into nine premultiplied 256-entry LUTs (dropping 18
68+
multiplies per pair) — ~1% gain, within noise; not worth the ugliness. The remaining
69+
regression is dominated by the six cbrt-LUT interpolations, which are inherent to OKLab.
70+
71+
### Euclidean vs HyAB (`|ΔL| + √(Δa²+Δb²)`, Abasi et al. 2019) — measured, decision pending
72+
73+
- HyAB is reported to track perceived *large* color differences (pixelmatch's regime) better
74+
than Euclidean, and separates the #127 cases slightly better (0.315 vs 0.247 for the "very
75+
different" pair).
76+
- Cost: +14% over Euclidean naively, reduced to ~4% with the sqrt-free thresholding below.
77+
- Since HyAB ≥ Euclidean for the same pair (L1 vs L2 norm), it is strictly more sensitive at
78+
the same threshold — its fixture counts land much closer to current YIQ behavior
79+
(fx4: 6942 vs YIQ's 6889, where Euclidean drops to 4760). Less churn for users' existing
80+
thresholds/baselines is an argument *for* HyAB despite the extra 14%.
81+
- **Decision criterion:** eyeball fixture diff outputs with both; pick HyAB if its extra
82+
sensitivity looks like signal rather than noise. Default leaning: HyAB — better literature
83+
support for large differences and smoother migration, at +24% total vs +19%.
84+
85+
## `brightnessDelta` (AA detection): keep gamma luma — measured
86+
87+
The AA detector uses a brightness-only delta (gamma-space Rec.601 luma). Should it become
88+
OKLab ΔL for consistency? **No.** Measured with a ΔL-based `brightnessDelta` variant:
89+
90+
- **Accuracy of the scalar itself:** gamma luma and OKLab L do disagree — mean |Δluma − ΔL|
91+
is 0.086 over random pairs (max 0.28), and pairs exist with equal luma but ΔL up to 0.055
92+
(e.g. dark red vs dark green). So ΔL is the *better lightness estimate* in the abstract.
93+
- **But AA detection doesn't need a perceptual lightness estimate.** It needs a cheap,
94+
consistent scalar to find the direction of the intensity ramp (darkest/brightest neighbor,
95+
equal-sibling counting). Along a real antialiasing ramp (blend between two edge colors),
96+
both scalars are monotonic and produce the same min/max neighbors.
97+
- **Measured impact of switching:** bench.js 956 → 1396 ms (**+46%**, because the detector
98+
calls `brightnessDelta` up to 16× per candidate pixel), and the AA-heavy fixture 5
99+
regressed 6 → 53 diff pixels (9x more AA false positives — ΔL's compression of dark-region
100+
contrasts breaks the darkest-neighbor selection there). Fixture 6 changed 38 → 29.
101+
- **Conclusion:** keep `brightnessDelta` on gamma luma (also keeps `drawGrayPixel` untouched).
102+
Cheap, and empirically better for this heuristic. Document the intentional inconsistency in
103+
a comment.
104+
105+
## AA algorithm review: is Vysniauskas 2009 still the right choice?
106+
107+
Current algorithm: a differing pixel is excluded as AA if, in either image, its 3×3
108+
neighborhood has ≤2 equal siblings, both a darker and a brighter neighbor exist, and the
109+
darkest or brightest neighbor sits in a flat region (3+ identical siblings) in *both* images.
110+
111+
Known failure modes (from the issue tracker and code reading):
112+
113+
1. **Thin shapes / 1px lines / rings** ([#74](https://github.com/mapbox/pixelmatch/issues/74)):
114+
a true AA pixel next to a 1px-wide stroke has no extreme neighbor with 3+ equal siblings,
115+
so it's counted as a real diff (false positive). Reported repeatedly for font smoothing.
116+
2. **Extreme-neighbor ties** (noted by a porter in #74): among equally-darkest (or brightest)
117+
neighbors only the first encountered is tested with `hasManySiblings`; a tied candidate
118+
that would pass is ignored → unstable, orientation-dependent results.
119+
3. **Gradients / noise / dithering:** `hasManySiblings` requires exact u32 equality, so AA on
120+
a smooth gradient background never finds a "flat" extreme → false positives. Conversely
121+
subpixel-AA color fringes (ClearType) differ in chroma more than luma and the
122+
brightness-only delta can under-detect the ramp.
123+
4. The `zeroes > 2` early-exit means a pixel on a flat-color boundary is never AA — usually
124+
right, occasionally wrong for 2px-wide AA ramps.
125+
126+
Options, cheapest first:
127+
128+
- **(a) Fix the tie handling:** collect all tied darkest/brightest neighbors and pass if any
129+
satisfies `hasManySiblings`. Small, local, strictly reduces false positives. Low risk —
130+
worth doing in the same major.
131+
- **(b) Relax the flatness requirement for one extreme** (proposed by the #74 reporter):
132+
require 3+ equal siblings for only one of darkest/brightest, with a weaker condition
133+
(e.g. 3+ *similar* siblings under some small OKLab delta) for the other. Directly targets
134+
the thin-stroke failure. Needs test fixtures (the #74 thread has example images) — worth an
135+
experiment, medium risk of admitting real 1px changes as AA.
136+
- **(c) Blend-ramp test:** a true AA pixel is a mixture of its two edge sides, so its color
137+
should lie near the OKLab segment between the darkest and brightest neighbors, in both
138+
images with consistent direction. More principled than sibling-counting and only runs for
139+
above-threshold pixels (cheap in aggregate). Novel — needs a fixture corpus to validate.
140+
- **(d) Shift-tolerance semantics** (resemble.js-style "does the other image have a matching
141+
pixel within 1px"): catches subpixel layout shifts, but changes what pixelmatch *means* —
142+
better as a separate opt-in option than a replacement, if ever.
143+
144+
**Recommendation:** don't replace the algorithm wholesale in this release — the metric change
145+
is the well-evidenced part; AA rework without a failure-case corpus is guesswork. Do (a) now
146+
(safe, fixes a real reported instability while we're breaking things anyway), prototype (b)
147+
and (c) against fixtures built from #74-style thin-stroke/text cases, and fold whichever wins
148+
into the same major if ready in time — otherwise ship the metric change alone.
149+
150+
## Prototypes
151+
152+
- `index-oklab.js` (repo root) — OKLab Euclidean `colorDelta`, cbrt LUT, `maxDelta = t²`.
153+
- `index-hyab.js` (repo root) — plain HyAB variant (`maxDelta = t`, sqrt per differing pixel).
154+
- `index-hyab-fast.js` (repo root) — HyAB with sqrt-free thresholding; `colorDelta` takes
155+
`maxDelta` and returns `0 / ±1`. Identical output to `index-hyab.js`, +24% vs YIQ.
156+
- Scratchpad (session): `index-oklab-dl.js` (ΔL brightness, rejected),
157+
`metrics.mjs` / `fastoklab.mjs` / `bench-all.mjs` / `bench-one.mjs` (all numbers above).
158+
159+
## Implementation steps
160+
161+
1. **Port the prototype into `index.js`**
162+
- Replace the YIQ math in `colorDelta`: blend against background if semi-transparent
163+
(produce blended colors rather than channel deltas), then sRGB→linear LUT → LMS matrix →
164+
cbrt LUT → OKLab distance (Euclidean or HyAB per the decision above).
165+
- `maxDelta = threshold * threshold` (Euclidean) or `threshold` (HyAB); black↔white = 1.0
166+
replaces the magic 35215. If HyAB: use the sqrt-free thresholding from
167+
`index-hyab-fast.js` (`colorDelta` takes `maxDelta`, returns `0 / ±1`).
168+
- Sign convention: negative when img2's pixel is brighter (ΔL > 0), as before with Y.
169+
- Module-load-time LUTs: `LIN` (256 × Float64) and `CBRT` (4098 × Float64).
170+
- `brightnessDelta` and `drawGrayPixel` stay on gamma luma (see above).
171+
2. **Fix the blending precision issue** (known prototype shortcut): blended channel values are
172+
fractional, but `LIN` is indexed by integers; the prototype truncates with `| 0`.
173+
Interpolate: `lin = LIN[i] + (LIN[i+1] - LIN[i]) * frac` (pad `LIN` to 257 entries), or
174+
evaluate the sRGB curve directly on the (rare) blended path.
175+
3. **AA fix (a):** handle darkest/brightest ties in `antialiased()`; add a regression fixture.
176+
4. **Recalibrate / regenerate fixtures**
177+
- Inspect diff outputs visually to confirm changes are improvements (fewer false positives
178+
on chroma-only deltas, new detections on dark-blue-vs-black cases); decide Euclidean vs
179+
HyAB here.
180+
- Regenerate expected diff PNGs and counts in `test/test.js`.
181+
- Add regression tests for the four #127 color pairs (small synthetic images; assert
182+
diff/no-diff at default threshold).
183+
5. **Benchmark:** run `bench.js` before/after in isolated processes; target ≤20% (Euclidean)
184+
/ ≤25% (HyAB sqrt-free) regression. If needed: try Float32Array LUTs, check `colorDelta`
185+
inlining. (Already ruled out: premultiplied LMS LUTs, per-color memoization.)
186+
6. **(time-permitting) AA experiments (b)/(c)** against a thin-stroke/text fixture corpus;
187+
include only if fixtures show clear improvement.
188+
7. **Docs and release**
189+
- README: document the new metric; note that `threshold` semantics changed and users may
190+
need to re-tune thresholds and regenerate stored diff baselines.
191+
- Update `colorDelta` JSDoc (cite Ottosson's OKLab post instead of Kotsarenko/Ramos).
192+
- Release as **semver-major**; respond to #127 (and #74 if AA fixes land) with the
193+
evidence tables.
194+
195+
## Open questions
196+
197+
- **Euclidean vs HyAB:** measured above; decide during fixture regeneration (step 4).
198+
Leaning HyAB.
199+
- **Default threshold:** 0.1 maps to similar sensitivity for achromatic deltas (gray Δ10:
200+
~0.034 OKLab vs ~0.038 YIQ) but Euclidean is less sensitive to large chroma-only deltas
201+
that YIQ over-reported. If HyAB is chosen this mostly resolves itself (counts stay close to
202+
current); if Euclidean, evaluate whether the default should drop to ~0.08.
203+
- **cbrt LUT size/accuracy:** 4096 entries → 2.6e-4 max distance error. Keep the LUT, or use
204+
`Math.cbrt` for simplicity (~40% slower colorDelta)? Leaning: keep the LUT.
205+
- **Alpha blending space:** blending stays in gamma sRGB (matches how browsers composite by
206+
default), then convert the blended color. Physically-correct blending would be in linear
207+
space. Leaning: keep gamma-space blending for compatibility with how images were rendered.
208+
- **How far to take AA changes in this major:** (a) is in; (b)/(c) depend on building a
209+
failure-case corpus (thin strokes, font smoothing — see #74 thread for example images).

‎README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,8 @@ const numDiffPixels = pixelmatch(img1, img2, diff, 800, 600, {threshold: 0.1});
2121

2222
Implements ideas from the following papers:
2323

24-
- [Measuring perceived color difference using YIQ NTSC transmission color space in mobile applications](https://www.spiedigitallibrary.org/conference-proceedings-of-spie/8011/80119D/Simple-perceptual-color-space-for-color-specification-and-real-time/10.1117/12.901997.full) (2010, Yuriy Kotsarenko, Fernando Ramos)
24+
- [A perceptual color space for image processing](https://bottosson.github.io/posts/oklab/) (2020, Björn Ottosson) — the OKLab color space used for color difference.
25+
- [Distance metrics for very large color differences](https://onlinelibrary.wiley.com/doi/10.1002/col.22451) (2019, Saeideh Abasi et al.) — the OKLab HyAB metric used to compare colors.
2526
- [Anti-aliased pixel and intensity slope detector](https://www.researchgate.net/publication/234126755_Anti-aliased_Pixel_and_Intensity_Slope_Detector) (2009, Vytautas Vyšniauskas)
2627

2728
## [Demo](https://observablehq.com/@mourner/pixelmatch-demo)

0 commit comments

Comments
 (0)