Skip to content

compress/flate: do not emit the preset dictionary into the output - #80539

Open
joechenrh wants to merge 2 commits into
golang:masterfrom
joechenrh:flate-dict-blockstart
Open

compress/flate: do not emit the preset dictionary into the output#80539
joechenrh wants to merge 2 commits into
golang:masterfrom
joechenrh:flate-dict-blockstart

Conversation

@joechenrh

@joechenrh joechenrh commented Jul 23, 2026

Copy link
Copy Markdown

When using a dictionary, fillWindow was not setting blockStart, which
caused a non-compressed block in first position to write the dictionary.
This mixup can happen because window stores both the look back window
and the block we are about to write. Set blockStart to skip over the
dictionary if we have to write the window.

Fixes #80538

@google-cla

google-cla Bot commented Jul 23, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@joechenrh
joechenrh force-pushed the flate-dict-blockstart branch from 7d27d7d to 1049404 Compare July 23, 2026 04:06
@joechenrh

Copy link
Copy Markdown
Author

@googlebot I signed it!

@joechenrh
joechenrh force-pushed the flate-dict-blockstart branch from 65ae4b0 to 9fedf56 Compare July 23, 2026 04:26
@gopherbot

Copy link
Copy Markdown
Contributor

This PR (HEAD: 9fedf56) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/804680.

Important tips:

  • Don't comment on this PR. All discussion takes place in Gerrit.
  • You need a Gmail or other Google account to log in to Gerrit.
  • To change your code in response to feedback:
    • Push a new commit to the branch used by your GitHub PR.
    • A new "patch set" will then appear in Gerrit.
    • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
    • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
    • Multiple commits in the PR will be squashed by GerritBot.
  • The title and description of the GitHub PR are used to construct the final commit message.
    • Edit these as needed via the GitHub web interface (not via Gerrit or git).
    • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
  • See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

@gopherbot

Copy link
Copy Markdown
Contributor

Message from Gopher Robot:

Patch Set 1:

(1 comment)


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

Message from Gopher Robot:

Patch Set 1:

Congratulations on opening your first change. Thank you for your contribution!

Next steps:
A maintainer will review your change and provide feedback. See
https://go.dev/doc/contribute#review for more info and tips to get your
patch through code review.

Most changes in the Go project go through a few rounds of revision. This can be
surprising to people new to the project. The careful, iterative review process
is our way of helping mentor contributors and ensuring that their contributions
have a lasting impact.

During May-July and Nov-Jan the Go project is in a code freeze, during which
little code gets reviewed or merged. If a reviewer responds with a comment like
R=go1.11 or adds a tag like "wait-release", it means that this CL will be
reviewed as part of the next development cycle. See https://go.dev/s/release
for more details.


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

NewWriterDict primes the compressor with fillWindow. That function
copies the dictionary into d.window and advances d.windowEnd and the
match index, but it leaves d.blockStart at 0.

When the first block is emitted, writeBlock passes
d.window[blockStart:index] to the huffman bit writer as the raw-bytes
fallback. Since blockStart is 0, this slice starts with the dictionary.
If the input is incompressible, storing the raw bytes is smaller than
the encoded tokens, so the writer emits them as-is, and the dictionary
ends up in the output. Decompressing then yields the dictionary
followed by the data.

Start the first block at the end of the dictionary instead, so the
fallback only sees real input. The dictionary still primes matches
through the hash chains, which do not use blockStart, so compression
of compressible data is unchanged.

Fixes golang#80538
@joechenrh
joechenrh force-pushed the flate-dict-blockstart branch from 9fedf56 to 478d270 Compare July 23, 2026 05:20
@gopherbot

Copy link
Copy Markdown
Contributor

Message from Jorropo:

Patch Set 1: Code-Review+2 Commit-Queue+1

(5 comments)


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

Message from golang-scoped@luci-project-accounts.iam.gserviceaccount.com:

Patch Set 1:

Dry run: CV is trying the patch.

Bot data: {"action":"start","triggered_at":"2026-07-23T05:11:22Z","revision":"7a5e2bb2dd878a6ecfef2e7d70f964c8f9ecab25"}


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

This PR (HEAD: 469f26b) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/804680.

Important tips:

  • Don't comment on this PR. All discussion takes place in Gerrit.
  • You need a Gmail or other Google account to log in to Gerrit.
  • To change your code in response to feedback:
    • Push a new commit to the branch used by your GitHub PR.
    • A new "patch set" will then appear in Gerrit.
    • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
    • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
    • Multiple commits in the PR will be squashed by GerritBot.
  • The title and description of the GitHub PR are used to construct the final commit message.
    • Edit these as needed via the GitHub web interface (not via Gerrit or git).
    • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
  • See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

@gopherbot

Copy link
Copy Markdown
Contributor

Message from Jorropo:

Patch Set 1: -Commit-Queue

(Performed by <GERRIT_ACCOUNT_60063> on behalf of <GERRIT_ACCOUNT_55763>)


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

Message from golang-scoped@luci-project-accounts.iam.gserviceaccount.com:

Patch Set 1:

This CL has passed the run


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

Message from golang-scoped@luci-project-accounts.iam.gserviceaccount.com:

Patch Set 1: LUCI-TryBot-Result+1


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

Message from chen joe:

Patch Set 2:

(4 comments)


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

Message from chen joe:

Patch Set 2:

(1 comment)


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

Message from Jorropo:

Patch Set 2: Auto-Submit+1 Code-Review+2 Commit-Queue+1

(1 comment)


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

Message from golang-scoped@luci-project-accounts.iam.gserviceaccount.com:

Patch Set 2:

Dry run: CV is trying the patch.

Bot data: {"action":"start","triggered_at":"2026-07-23T15:59:15Z","revision":"4b068081d0a0d083e494ab31fc54f85761076a44"}


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

Message from Jorropo:

Patch Set 2: -Commit-Queue

(Performed by <GERRIT_ACCOUNT_60063> on behalf of <GERRIT_ACCOUNT_55763>)


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

Message from golang-scoped@luci-project-accounts.iam.gserviceaccount.com:

Patch Set 2:

This CL has passed the run


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

Message from golang-scoped@luci-project-accounts.iam.gserviceaccount.com:

Patch Set 2: LUCI-TryBot-Result+1


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

@gopherbot

Copy link
Copy Markdown
Contributor

Message from Cherry Mui:

Patch Set 2: Code-Review+1


Please don’t reply on this GitHub thread. Visit golang.org/cl/804680.
After addressing review feedback, remember to publish your drafts!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

compress/flate: Writer with a preset dictionary emits the dictionary into the output stream

2 participants