Skip to content

fix: stop the codecs losing, mis-parsing and over-allocating - #49

Merged
IAmJSD merged 10 commits into
Infrawrench:mainfrom
ProdigyRahul:combine/codec
Aug 27, 2026
Merged

fix: stop the codecs losing, mis-parsing and over-allocating#49
IAmJSD merged 10 commits into
Infrawrench:mainfrom
ProdigyRahul:combine/codec

Conversation

@ProdigyRahul

@ProdigyRahul ProdigyRahul commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

groups eight branches: what the psd and affinity codecs lose, mis-parse or refuse to admit.

data silently dropped on save

  • four classes of psd block were read and regenerated as empty: global layer mask info, the document-level additional-info blocks (Patt patterns, lnk2 linked smart objects, Txt2, FMsk) and per-layer blending ranges ("Blend If"). open a photoshop file with any of them, save, and they were gone — while the readme says every block is preserved byte-for-byte.
  • smart objects lost their source pixels. the payload rides on Layer::smart, which the writer never serialized, so after save-and-reopen a smart object was a plain raster of its last rasterization and every further transform degraded it. now written to a private ScSo block, the same trick the type tool plays with PsTx.
  • adjustment layers made or edited in schist were written as empty rasters: their settings lived only in params_json, which no psd reader understands.
  • layer fill opacity was never read and never written. a file with Fill 0% opened fully opaque, and editing Fill saved the file's original value back over it.

parsing that desynchronised or over-allocated

  • descriptor parsing consumed an unknown value's tag but not its payload, then read that payload as the next key — silent garbage, worst inside a nested descriptor whose parent kept going.
  • the affinity object graph took an element count straight from the file.
  • the psd writer accepted layer counts and names it could not encode.

depth through the non-psd codecs

(icc profiles for png and jpeg landed separately in #48; what is left here is webp and tiff, plus depth.)

  • every non-psd import was forced to 8 bits, so a 16-bit scan lost half its precision on the way in. ExportOptions::bit_depth was consulted only to pick the dither level, never to choose an output depth.
  • exporting a transparent document to jpeg produced a black background rather than matting it.

what consolidation caught

merging the icc work with the bit-depth work meant routing 16-bit export through ImageEncoder::write_image(&[u8], ..), which reads samples back as native-endian. the first attempt wrote big-endian and every sample came out byte-swapped — invisible in either branch alone, caught by export_honours_the_requested_bit_depth.

one thing left alone

the audit also says 16-bit psd samples should be normalised over 0..=32768 rather than 0..65535. the psd reader's own comment argues the opposite, and only a photoshop-authored fixture settles it; guessing halves or doubles the brightness of every 16-bit file. the false claim in crates/color that a conversion happens at the codec boundary is fixed; the numbers are untouched and the open question is documented.


cargo fmt --all --check, cargo clippy --workspace --all-targets -D warnings, cargo test --workspace — 651 passed, 0 failed.

how this relates to my other open prs

these seven are independent of each other — each branches off main and each is green on its own. they do share files with my five open prs (#36, #38, #42, #44, #46), mostly workspace.rs, so whichever lands first will leave the others needing a rebase. happy to rebase in whatever order suits you, or to split any of these further if one is too big to review in a sitting.

@ProdigyRahul
ProdigyRahul marked this pull request as ready for review August 26, 2026 12:25

@IAmJSD IAmJSD left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review was written by Claude (Fable 5), acting on Astrid's behalf.

This is genuinely good codec work — every commit's claim checks out, the tests are real round-trips through the public read/write API, the descriptor/affinity hardening is strictly safer than the old desync-and-truncate behaviour, the adjustment-layer encoder is a tested true inverse of the parser, and the 16-bit import/export and probe fixes (WebP-in-RIFF, BigTIFF) are correct. One blocker and one major keep it from merging as-is:

Blocker — preserved-block padding mismatch loses the blocks it preserves (crates/codec-psd/src/writer/mod.rs:290): the writer pads each document-level preserved block with b.pad_to(4) — alignment to the absolute file offset — while the reader (reader/layers.rs:130) skips (4 - len % 4) % 4 bytes relative to the payload length. They disagree whenever a block doesn't happen to start 4-aligned, which depends on layer-name lengths, channel data sizes, and the (now-preserved, possibly odd-length) global layer mask — so real files hit it constantly. Reproduced: a doc with two preserved blocks (Patt, lnk2) and a layer named "ab" (whose luni shifts alignment by 2) round-trips to just [Patt] — the reader's tolerant trailing-garbage break silently discards everything after the first block, and Photoshop would misparse the stream the same way. Fix is two lines: emit (4 - len % 4) % 4 pad bytes per block, as the per-layer extras loop already does relatively.

Major — smart-object sources are quantized to 8-bit (crates/codec-psd/src/smart.rs:59): write_smart serializes via pixel(x, y).to_u8(), so a 16-bit image placed as a smart object in a 16-bit document silently loses 8 bits per channel on save/reopen — contradicting both the block's purpose ("keep their source pixels") and this PR's own bit-depth commit, which fixes exactly this loss class in the importers.

Smaller items:

  • Preserved blocks always re-emit an 8BIM signature, but the reader accepts 8B64; an 8B64 block with a key outside PSB_U64_KEYS is read with a u64 length and re-written with a u32 one — not byte-exact for PSBs, and data.len() as u32 would silently truncate a >4 GiB block.
  • The PR body says the TIFF ICC profile "is asserted on the bytes" in a test — no such test exists in this diff (the encoders do implement set_icc_profile, so the code works, but the claim is unbacked).
  • Value::Unknown in psd-descriptor is now a never-constructed dead variant; remove it.
  • The old blit_rgba8 doc comment now dangles above blit_rgba_f32 (core/src/document.rs:1024), stale RGBA8 sentence included.
  • Commit 5c002f6 bundles the unrelated (correct) colormgmt soft-proof fix with the PSD fill-opacity fix, and the body's "png/jpeg dropped icc profiles entirely" describes #48, already on main.

With the padding fix and the smart-object depth either fixed or explicitly accepted as a known limitation, this is a merge. Happy to re-review quickly.

@ProdigyRahul

Copy link
Copy Markdown
Contributor Author

fixed

@IAmJSD IAmJSD left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed by Claude (Fable 5) on Astrid's behalf. All findings addressed: the preserved-block padding is now relative and symmetric with the reader (with an alignment-sweep test across all four residues — nice), smart-object sources round-trip losslessly via the v2 f32 payload with v1 still readable, the 8B64/u64 truncation path is fixed, the TIFF/WebP ICC assertions exist, the dead variant and doc-comment splices are gone, and the soft-proof revert is byte-identical to main (the fix now lives in #50 as advertised). Two non-blocking notes for later: the smart-object decompress bound's "guard against gigabytes" comment is optimistic now that v2 samples are 16 bytes (~6.4 GB transient worst case from a hostile zlib payload), and storing f32 even for 8-bit sources quadruples the block on disk — the version field makes a per-depth width a compatible future change. Merging.

@IAmJSD
IAmJSD merged commit a8529b0 into Infrawrench:main Aug 27, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants