Skip to content

Conversation

@squeek502
Copy link
Member

Previously, the logic in peekDelimiterInclusive (when the delimiter was not found in the existing buffer) used the n returned from r.vtable.stream as the length of the slice to check, but it's valid for vtable.stream implementations to return 0 if they wrote to the buffer instead of w. In that scenario, the indexOfScalarPos would be given a 0-length slice so it would never be able to find the delimiter.

This commit changes the logic to assume that r.vtable.stream can both:

  • return 0, and
  • modify seek/end (i.e. it's also valid for a vtable.stream implementation to rebase)

Also introduces std.testing.ReaderIndirect which helps in being able to test against Reader implementations that return 0 from stream/readVec

Fixes #25428


Before this PR, the added "takeDelimiterInclusive on an indirect reader when it rebases" test would fail with error.StreamTooLong since it'd always be searching for the delimiter in a 0-length slice.


This is a very targeted fix. It does not address any of the following:

It also assumes that the stream ambiguity described in #25170 should be resolved in the "writing to w cannot modify the buffer" way (i.e. this fix assumes that no implementation will try to write to both buffer and w in the same stream call).

(see also #25169 for some extra context around those issues)


The added "readSliceShort with indirect reader" test is just to verify the std.testing.ReaderIndirect implementation around rebasing within stream (without the rebase within stream, that test would enter an infinite loop).

@kaizoplant
Copy link

kaizoplant commented Oct 6, 2025

ive been trying out this implementation, and i was wondering about this line

if (std.mem.indexOfScalarPos(u8, r.buffer[r.seek..r.end], existing_buffered_len, delimiter)) |delimiter_index| {
     return r.buffer[r.seek .. delimiter_index + 1];
}

i think if im not incorrect, what should be returned is

r.buffer[r.seek .. r.seek + delimiter_index + 1]

since if seek is non-zero, then the delimiter will be at the wrong spot
alternatively, i think the following also works

if (std.mem.indexOfScalarPos(u8, r.buffer[0..r.end], r.seek + existing_buffered_len, delimiter)) |delimiter_index| {
    return r.buffer[r.seek .. delimiter_index + 1];
}

idk if any of those would be particularly better or worse but yeah

@squeek502
Copy link
Member Author

squeek502 commented Oct 6, 2025

Good catch. Will try to alter the test so that it fails with my broken version.

EDIT: Actually, it's not really feasible to construct a failing test, or at least without using a vtable.stream implementation that advances the seek position (which seems like it should violate the API, see #25170 for more on that)

Note also that involving r.seek here instead of assuming it is 0 is mostly an attempt at future-proofing depending on how #25103 is resolved.

…ns that return 0

Previously, the logic in peekDelimiterInclusive (when the delimiter was not found in the existing buffer) used the `n` returned from `r.vtable.stream` as the length of the slice to check, but it's valid for `vtable.stream` implementations to return 0 if they wrote to the buffer instead of `w`. In that scenario, the `indexOfScalarPos` would be given a 0-length slice so it would never be able to find the delimiter.

This commit changes the logic to assume that `r.vtable.stream` can both:
- return 0, and
- modify seek/end (i.e. it's also valid for a `vtable.stream` implementation to rebase)

Also introduces `std.testing.ReaderIndirect` which helps in being able to test against Reader implementations that return 0 from `stream`/`readVec`

Fixes ziglang#25428
@andrewrk andrewrk merged commit 328ae41 into ziglang:master Oct 8, 2025
9 checks passed
@daurnimator
Copy link
Contributor

daurnimator commented Oct 23, 2025

@squeek502
Copy link
Member Author

@daurnimator I think you might be running into #25597 (comment)

(side note: the changes in this PR were obsoleted by #25512 where peekDelimiterInclusive was reworked)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

std.crypto.tls.Client hangs on reading application data

4 participants