perf(streams) astubbs#242: the FFI fast-path spike - sub-microsecond crossings, and a mis-specified floor

STRATEGY.md

@@ -259,6 +259,37 @@ single-session transport, bounded by the untested premise that windowing is not
 practice. What would reopen it is a transport that cuts the per-crossing cost by roughly two orders
 of magnitude - the embedded/FFI direction - not any windowing or placement design.
 
+**Updated 2026-08-25 (second), and the reopening condition above has been met - but what it
+reopened is the floor, not the verdict.** (The crossings:
+`docs/inflight/perf-crossing-cost-ladder.md`; the native engine:
+`docs/inflight/perf-streams-under-native-image.md`; the floor decomposition and the in-session
+retake: `docs/inflight/perf-streams-engine-floor.md`.) The paragraph above named its own condition
+for reopening - a transport cutting the per-crossing cost by roughly two orders of magnitude - and
+that condition is now met: GraalWasm crosses in **747ns** and a Numba `@cfunc` in **19.9ns**
+against the **135us** fitted here, and the Streams engine has been proven to run as a native-image
+binary. Two results followed, pointing opposite ways. **For the wrapper:** most of the measured
+engine floor turned out to be an instrument choice rather than Kafka Streams - the state-store
+cache was set to zero - and with the cache on and nothing crossing, the wrapper reaches **69,265
+rec/s** at hopping-12 and **169,748 rec/s** at tumbling. **Against it:** retaken with the
+reimplementation arm interleaved in the *same* session, the dictionary still wins **4.70x** at
+tumbling and **6.64x** at hopping-12. So removing the crossing does not by itself close the gap,
+and the 69x/122x above should be read as the crossing's price rather than the residual. (Both
+retaken figures are firm: the reimplementation arm's apparent instability was traced to a stalled
+consumer fetch path, not to the arm, and the retake ran below the record count where that stall
+occurs. The **122x** above is itself an artefact of that stall and understates the gap.)
+
+**The substantive change is neither number: it is that the floor was mis-specified.** The
+reimplementation arm is a stateless, non-durable dictionary - no store, no changelog, no restore,
+no rebalance recovery, no exactly-once - which makes it the floor for a product Kafka Streams is
+not in the business of being. Measuring against it answers *"can a toy beat an engine at toy
+work"*, and it can, at any transport speed; sharpening that number further decides nothing. **The
+question that decides this strategy is the crossover: how many of the features a user actually came
+for can be added back to that dictionary before hand-rolling becomes the worse choice.** That is
+the measurement now under way, one feature at a time, starting with durability. Until it returns,
+the not-offered verdict above stands on its own evidence - but the *reason* recorded for it, that a
+reimplementation is simply faster, is too strong. It is faster only at the feature level where
+nobody needed Kafka Streams in the first place.
+
 So the hard part of Streams-in-another-language is not the streaming; it is that a topology has no
 portable description. ~~That is an IDL to design~~ - **and the proof of concept showed it is not.**
 See below.

docs/inflight/branch-crossing-cost-ladder.md

@@ -0,0 +1,50 @@
+# The FFI fast-path spike: the crossing-cost ladder, and the feature-crossover ladder
+<!-- post-merge: checked -->
+
+<!-- inflight-type: task -->
+<!-- inflight-impact: coordination -->
+
+Cut 2026-08-25 from astubbs#334's head (post-windowing-verdict), on the owner's direction, to run
+the spike the bet-off named as its reopening condition.
+[`perf-embedding-the-engine-over-ffi.md`](perf-embedding-the-engine-over-ffi.md) **owns the
+candidate** - the compile-the-function design, its prior-art survey, the owner's
+WASM-subset-is-fine direction, and the one-C-ABI-seam refinement; what is here is only what the
+BRANCH is for.
+
+**The question:** what does one host-function crossing cost under each in-process mechanism, and
+does any clear the pre-registered bar of **~1.35us marginal** (two orders under the measured 135us
+gRPC crossing)? Arms: (a) the gRPC figure as context (measured, astubbs#334), (b) an in-process
+queue handoff (the embedded pull seam's shape), (c) a raw C-ABI call (ctypes/FFM into a no-op and
+a realistic fold), (d) a Numba `@cfunc` pointer called from the engine side, (e) a GraalPy
+polyglot call, (f) a **GraalWasm** UDF - (f) primary per the owner's direction, and a GraalWasm
+result alone green-lights a product slice. Method inherits the windowing spike's discipline:
+predictions in the tree before runs, an instrument check that can move (a busy-wait injection must
+show up), a realistic-fold arm beside every no-op (whether the fast path covers real workloads,
+not benchmarks), and the end-to-end ceiling restated against U6's engine floor (free crossings
+reach arm D's rate, not arm H's).
+
+**Named companion gap, owner-corrected 2026-08-25 - Kafka Streams has never run under GraalVM
+here.** The embedded `--shared` library covers PC core only; the Streams fast path's end state
+needs one of two unproven routes: (1) native-image including Kafka Streams (RocksDB JNI,
+reflection over serde config, unknown metadata surface), or (2) **libjvm embedding** - a full JVM
+hosted in the client process (JIT retained, no native-image build, heavier footprint). The ladder
+measures call mechanics with a minimal harness, so neither route gates the measurement - but a
+green ladder without settling this gap is not a green light for Streams-in-process, and the
+write-up must say which route it assumes. **Probed 2026-08-25, gap crossed for the in-memory
+surface**: route (1) is proven and cheaper - the engine builds into a 78MB native binary that
+passes the demo, one traced capture was the whole wall, and libjvm demotes to fallback;
+[`perf-streams-under-native-image.md`](perf-streams-under-native-image.md) owns the result and its
+durability boundary.
+
+**The spike carries a second ladder, in the other dimension: the feature-crossover ladder.** The
+transport ladder above asks how cheaply a call can cross the boundary. That question is answered,
+and the answer did not settle the strategy - so the spike also adds the features a user actually
+came for back to the *reimplementation*, one at a time, to find where hand-rolling becomes the
+worse choice. Durability is the first rung, measured; exactly-once is the candidate for the
+second. [`perf-streams-engine-floor.md`](perf-streams-engine-floor.md) **owns both sets of
+numbers** - what is here is only that the spike has two purposes, so a reader arriving at the
+transport ladder does not take it for the whole.
+<!-- post-merge: checked -->
+
+Delete this note when the branch lands or is superseded; the spike's results note (created on this
+branch) will carry the numbers.