Skip to content

chore: small improvement and optimization in the normalization process#61

Open
doudou wants to merge 3 commits intomasterfrom
small_opt
Open

chore: small improvement and optimization in the normalization process#61
doudou wants to merge 3 commits intomasterfrom
small_opt

Conversation

@doudou
Copy link
Member

@doudou doudou commented Feb 26, 2026

No description provided.

The report would use the compressed size as total and the uncompressed for
progress, which rather obviously does not work.

The only efficient way is to count compressed bytes, which is good enough really
… buffer is empty

If the zstd lib does not have enough data for a full block, it returns an empty
buffer. Bypass the underlying IO's write call in that case.
Was 1M, which turns out to be rather expensive if the samples are small. 128k
seems to be a good tradeoff
@doudou doudou requested a review from wvmcastro February 26, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants