client: take an async Stream for client-streaming requests

[iainmcgin]

Summary

Client-streaming calls now take an async Stream of requests instead of a synchronous iterator.

call_client_stream and the generated client-streaming methods accept impl ClientRequestStream<Req> — a sealed trait implemented automatically for every Stream<Item = Req> + Send + 'static — so request messages can be produced as they become available — paced by timers, read from sockets, forwarded from channels — without buffering the whole upload or blocking a thread. The synchronous-iterator shape could not express this: there was no way to await between messages, which ruled out pump-style uploads (e.g. forwarding chunks under a per-chunk idle deadline).

Breaking change, targeted at 0.9.0. Migration for a ready collection is one line, with no new dependency:

// before
client.sum(requests).await?;
// after
client.sum(connectrpc::client::stream_iter(requests)).await?;

Stream (the futures 0.3 trait) and stream_iter (futures::stream::iter) are re-exported from connectrpc::client, and every generated client-streaming method carries the migration hint in its rustdoc. The bound trait carries #[diagnostic::on_unimplemented], so passing a collection directly produces compiler output that names stream_iter and explains the Send + 'static requirement, rather than a bare unmet-trait error.

Design: the stream is the request body

The stream is not pumped by a library-side drain loop — it backs the HTTP request body directly (EncodingBody, which envelope-encodes each message in poll_frame). The transport polls for the next frame only while it can send, so:

• backpressure is HTTP/2 flow control (peak memory ~one envelope, previously a 32-envelope channel);
• a server that ends the RPC mid-upload produces a response and the call returns, even while the caller's stream is idle — previously unobservable until the deadline;
• a server that sends response headers early while continuing to read the upload keeps receiving messages (a drain loop racing the response future would truncate here — caught by review and pinned with a regression test).

An encode failure is emitted through the body (aborting the request) and mirrored into the call's return value so the caller sees the precise error. The stream is held in a sync_wrapper::SyncWrapper (new tiny zero-dep dependency, the axum pattern) so the boxed body is Sync without requiring Sync of the caller's stream. The previous channel + detached-send-task machinery for client-streaming is deleted; bidi is unchanged (its push-style send() API keeps the channel).

Changes

• connectrpc/src/client/mod.rs: signature, EncodingBody, Stream/stream_iter re-exports, rewritten docs with # Errors.
• connectrpc-codegen/src/codegen.rs: generated arg type + doc hint; checked-in generated dirs regenerated.
• Call sites updated: conformance client, benches, streaming-tour example, e2e tests; docs/guide.md shows both stream_iter and a channel-backed producer.
• New e2e tests: async producer paced by a channel; early-response-headers-do-not-truncate-upload; early-server-response-unblocks-pending-stream.

Testing

• task lint / task test green; clippy all-targets -D warnings clean; docs with broken-links denied; MSRV (1.88), wasm32, and minimal-features checks pass.
• Conformance client suites: Connect 2580, gRPC 1454, gRPC-Web 2838 — all passing.
• Live socket drives (HTTP/1.1 and HTTP/2): a producer pacing 3 messages 100 ms apart aggregates correctly (elapsed ≥ 300 ms proves incremental sending); a handler rejecting without draining returns in ~2 ms against a never-yielding stream (previously an unbounded hang).
• cargo semver-checks vs 0.8.1 passes (the break is in an impl Trait argument bound, outside its lint coverage); declared in the changelog fragment.

[iainmcgin]

[claude code] Benchmarked the stream-as-body design against the previous channel-based client-streaming (base 4da239d) to answer whether the one-envelope-at-a-time body needs a buffer. Two independent criterion compare runs on a dedicated bare-metal host (c7i.metal-24xl, no shared tenancy), baseline and candidate binaries interleaved on the same box; untouched paths (unary/small, server_stream) as drift controls, which stayed ≤ ~1.7% throughout. Positive = this PR slower.

bench	run 1	run 2
client_stream (10 small msgs) — connect / grpc / grpc-web	−2.0% / −2.2% / −2.0%	−1.2% / −1.6% / −0.9%
client_stream_many_small (1000 msgs) — connect / grpc / grpc-web	−3.5% / −2.0% / −3.2%	–
client_stream_large (10 × 1 MiB) — connect	+9.4%	+6.1%
client_stream_large — grpc	p=0.77 (±13%)	p=0.20 (±10%)
client_stream_large — grpc-web	−6.2%	+5.8%

Reading:

• Small-message streaming got 1–3.5% faster, consistent across six benches and both runs: the per-message cross-task hop (bounded channel + wakeups + oneshot bridge) is gone, and the transport pulls frames directly. No buffering needed to keep the wire busy — HTTP/2 flow-control windows and socket buffers already do that.
• 1 MiB-message uploads over HTTP/1.1 + Connect regressed ~6–9%, reproducibly. The magnitude matches lost encode/send overlap: each message costs ~2 MiB of encode+envelope memcpy that previously ran on the caller's task concurrently with the send task's socket writes, and now serializes on one task. The other two large benches are dominated by allocator bimodality at 1 MiB payloads (grpc-web flipped sign between runs; grpc never reached significance).
• The regression's substance is exactly the copy work already queued for 0.9.0: server: pass large response payloads through by refcount instead of copying #219's refcount-chained framing removes the envelope memcpy for ≥16 KiB payloads, and the upstream encode work removes the encode-side copy. Re-running client_stream_large stacked on server: pass large response payloads through by refcount instead of copying #219 should confirm most of it comes back.

The two new bench groups (client_stream_many_small, client_stream_large) are added in this PR so the comparison stays reproducible.

[iainmcgin]

[claude code] Rebased onto main via merge: #226's abandonment fix targeted the channel-era detached send task, which this PR deletes — dropping the call future now drops the transport send and request body directly, so the guarantee holds structurally. All five #226 tests are ported to the stream API (the mid-drain backpressure case becomes an unread-request-body case, since there is no drain loop to park in), the # Cancellation rustdoc is kept with the mechanism description updated, tokio/macros stays consolidated on the base dependency (comment narrowed to the server accept loop), and the #226 changelog fragment is reworded to describe the behavior rather than the superseded mechanism. Full battery re-run post-merge: 562 all-features unit tests, Connect client conformance 2580/2580, lint/docs/MSRV/wasm/minimal clean.

[lovesegfault]

Took a close look at the redesign — really nice work; no correctness or safety concerns from me. Comments below are all non-blocking: a few doc notes on correct-but-surprising behavior, plus small polish ideas.

[lovesegfault]

This covers the early-Ok race, but there's a second direction: with a background-driven body, an encode error can land after this check and is then never read — the call returns Ok for a truncated tail. I think that's fine (the server already finished its response), but a sentence saying it's intentional would help a future reader.

[lovesegfault]

Could we add a cancellation note here? This is the doc users actually read, and dropping the returned future (e.g. a timeout firing) now silently drops unsent messages. Something like: "Dropping the returned future cancels the call; unsent messages are never delivered." A matching sentence in the guide would be nice too.

[lovesegfault]

Worth a sentence here on panics: the stream is polled on the transport's task now, so a panic in poll_next surfaces as a generic transport error instead of propagating (and can take down a shared connection). Would save someone a confusing debug session.

[lovesegfault]

Might as well add:

fn is_end_stream(&self) -> bool {
    self.done
}

The default is always false, so downstream layers need an extra poll_frame to notice exhaustion — done already means exactly this.

[lovesegfault]

Nothing forces a construction site to also drain the mirror — the pairing is convention only. Maybe EncodingBody::new(...) -> (Self, EncodeErrorSlot) with a #[must_use] slot? Small change, makes the contract compiler-visible.

[lovesegfault]

Style nit, take it or leave it: the other call_* fns name all their generics; S: ClientRequestStream<Req> would match them and show the bound in rustdoc. (Not about turbofish — that already works; naming it would just add a _ to the conformance call.)

[lovesegfault]

Small one: we re-export UnaryResponse/ServerStream/BidiStream at the crate root, but not stream_iter, which most callers will want. pub use client::stream_iter; in lib.rs would shorten the common call site.