perf(images): encode embedded PNGs with adaptive filtering, not NoFilter - #908
perf(images): encode embedded PNGs with adaptive filtering, not NoFilter#908ajbufort wants to merge 2 commits into
Conversation
to_png_bytes() encoded every non-JPEG image (Flate/CCITT/JBIG2/JPX) with CompressionType::Fast + FilterType::NoFilter. NoFilter disables PNG's per-scanline Sub/Up/Average/Paeth filtering - exactly the step that lets deflate compress smooth/scanned content - so the PNG stream stayed near-raw. A full-page scanned background that the source PDF stores in ~110 KB was re-emitted at ~4.4 MB (~40x), and any consumer inlining these images (e.g. base64 data URIs) inherited the blow-up. Switch to CompressionType::Default + FilterType::Adaptive. This is purely a container-size choice - the decoded pixels are byte-identical (PNG is lossless) - trading a little encode CPU for large size wins. Measured over a sample of image-heavy documents: total inlined-HTML output 2200 MB -> 928 MB (0.42x); individual flat-scan docs up to 83x smaller; pixel-fidelity and text metrics byte-identical (max delta 0.0000). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Anthony J. (Tony) Bufort <ajbufort@ajbconsulting.us>
|
Thanks @ajbufort — this is a clean, well-reasoned change, and the write-up is exactly the right level of detail. I verified it end-to-end and it holds up strongly. First, the red CI is a false alarm — it's base staleness, not your code. I merged your branch onto current Verification of the claim (built
And I confirmed the lossless half directly — extracted the embedded PNG from both outputs on one doc (8,799,699 → 502,897 bytes), decoded both, and the pixels are byte-identical: ImageMagick So the substance is a clear win. Before it can merge, the standard process items — same bar every recent PR went through:
(DCO sign-off is already there — thanks for that.) Rebase + those items and I'll review promptly; the technical case is already made. |
yfedoseev
left a comment
There was a problem hiding this comment.
Approving on the technical merits — I verified it end-to-end (see my detailed comment above): compiles clean on current main, all 65 image tests pass, and every embedded PNG across the scanned test docs is byte-identical after decode (zero information loss) while the container shrinks 17–150×. Solid, well-scoped change.
Before this can actually merge, three items still need to land per CONTRIBUTING.md — the same bar every recent PR followed:
- Sign the CLA (currently the hard merge blocker) — comment exactly this line, nothing else:
I have read the CLA Document and I hereby sign the CLA - Add a fails-on-revert regression test (assert Adaptive output is materially smaller than NoFilter + decoded-pixel identity).
- Link an accepted issue (
Closes #NNN) and disclose AI assistance (tool + extent, or "none").
Rebase on main to clear the stale-base CI. The review's done — these are the remaining gates.
|
I have read the CLA Document and I hereby sign the CLA You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot. |
Problem
PdfImage::to_png_bytes()encodes every non-JPEG image (Flate / CCITT / JBIG2 / JPX) with:FilterType::NoFilterdisables PNG's per-scanline Sub/Up/Average/Paeth filtering — the exact step that lets deflate compress smooth / scanned content — so the stream stays near-raw. A full-page scanned background that the source PDF stores in ~110 KB (FlateDecode) re-encodes at ~4.4 MB (~40×). Any consumer that inlines these images (e.g. base64 data URIs in an HTML export) inherits the blow-up; on a sample of image-heavy documents this alone accounted for hundreds of MB.Fix
Switch to
CompressionType::Default+FilterType::Adaptive. This is purely a container-size choice — the decoded pixels are byte-identical either way (PNG is lossless) — trading a little encode CPU for large size reductions.Measured
Over a sample of image-heavy documents, total inlined output 2200 MB → 928 MB (0.42×); individual flat-scan docs up to 83× smaller; pixel-fidelity and text metrics byte-identical (max delta 0.0000). The re-encode cost is per-image-decode, not per-render.
Full gate on this branch off
main:cargo test --lib(5750 passed),cargo test --doc(141 passed),cargo clippy --libclean,cargo fmt --checkclean.