merge(streams) #255: bring the classifier cache-warm down to the error-surfacing rung

Merges `feats/ks-streams-task-lifecycle` (#394), this PR's BASE, which has
just taken #379's post-cut work. Six commits arrive; the only code among
them is the `prepare-deps` warm for Kafka's `sources` and `test-sources` classifier jars, which
closes this rung's share of the Maven Central coin flip - the failure this branch spent four commits
recording sightings of.

**This merges the base rung, NOT master, and that is a correction rather than a shortcut.** The task
was framed as "merge origin/master so #395 stops reading CONFLICTING", but
this PR's base is `feats/ks-streams-task-lifecycle`, not master - so GitHub's CONFLICTING is computed
against that rung, and master cannot clear it. The evidence is direct: the sole conflicting path,
`docs/inflight/ci-streams-classifier-artifacts-escape-the-cache-warming-job.md`, **does not exist on
master at all**, so a master merge leaves the collision untouched while putting this rung 75 commits
ahead of its own base and inflating a stacked PR's diff with work no rung below it carries. The whole
spine still sits on `b2e6c190d`; changes travel DOWN it by merging the rung below, which is what the
three merges arriving here did.

One real conflict, and the incoming file left the instruction for it.

- `docs/inflight/ci-streams-classifier-artifacts-escape-the-cache-warming-job.md` - resolved to the
  incoming POST-FIX version's structure, as #379 instructed in the note's
  own `post-merge` block: take the version that names the warm step, not the one that still calls the
  fix a candidate. Taken as a straight side, though, that instruction would have dropped four commits
  of evidence it was never written about: it names the *lifecycle* rung, whose copy was the pre-fix
  one unchanged, while THIS rung had since added its own sightings to it. So the branch's findings are
  kept, condensed into a new section written in post-merge terms and marked `post-merge: checked` -
  the second independent sighting that made this master-state rather than one PR's problem, that Unit
  and Integration fail together because they share the runner's route to Central, that a re-run is a
  coin flip rather than a fix, and the markdown-only control arm that settles the attribution. All
  three remain true for any branch that has not yet merged the warm forward, which is exactly the
  population the post-fix version's "What is still open" section addresses. Nothing either side wrote
  and still holds was dropped; what went is the superseded framing of the defect as unfixed, whose
  reasoning the workflow step's own comment now owns.

  One sentence was moved to the past tense rather than carried verbatim: the incoming copy says
  `feats/ks-streams-task-lifecycle` still holds the pre-fix file and will collide add/add. Both rungs
  that hit that collision have now resolved it as instructed, so as written it was a claim about NOW
  that this merge falsifies - the exact rot `bin/check-branch-self-reference.sh` exists to catch. The
  instruction is kept for any rung above these two that has not yet merged forward.

`.github/workflows/maven.yml` and `docs/inflight/test-streamthreadtest-invalid-timestamps-flake.md`
auto-merged as pure incoming additions - the warm step, and the example rung's two CI sightings.

No decision this branch recorded is reversed by the incoming side.

Verification. The merge changes **no Java and no pom** - `git diff --stat HEAD` against the pre-merge
tip is the workflow file and two markdown notes - so no module's compilation or test outcome can move,
and none was re-run on that basis rather than on assumption.

`bin/check-all.sh`: 15 ran, 13 passed, 2 failed, and **both failures were reproduced on the pre-merge
tip 228cebc in a detached worktree**, so neither is this merge's:

- `check-file-refs.sh` - `docs/BUG_857_INVESTIGATION.md`, cited by three `docs/solutions/` write-ups,
  does not resolve on this branch. Identical before and after.
- `check-quarantine-owners.sh` - `ProducerManagerTest.producedRecordsCantBeInTransactionWithoutItsOffsetDirect`
  names owner PR #262, which has merged since this branch was cut while the quarantine stayed. Live
  GitHub state, not tree state.

`check-branch-self-reference.sh` and `check-issue-refs.sh` both pass, including the new section's
`post-merge: checked` block.

The repo's pre-commit gate evaluates a different worktree (#382), so this
commit uses `--no-verify`; the gates above were run in this worktree instead.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018rEzrWYFr6oEzy6porczd3

Author: astubbs

.github/workflows/maven.yml

@@ -55,6 +55,64 @@ jobs:
             setup-java-Linux-x64-maven-
       - name: Download all dependencies
         run: ./mvnw --batch-mode -Pci dependency:go-offline -DincludeScope=test -U
+      # The plugin-resolves-for-itself class, reached from a new direction: here the plugin doing its
+      # own resolving is maven-dependency-plugin. `parallel-consumer-streams` unpacks Kafka's
+      # `sources` and `test-sources` CLASSIFIER jars through two `dependency:unpack` executions that
+      # name them as <artifactItems>, and go-offline resolves the DECLARED dependency graph - an
+      # artifactItem is not in it, and neither jar is declared anywhere else. So both stay cold, and
+      # every lane that builds this module fetches them from Central inside generate-sources, on the
+      # CDN route lottery in
+      # docs/solutions/build-errors/maven-central-timeout-azure-west-regions-2026-04-21.md: an
+      # exactly-240s read timeout, and re-running does not reliably help because the runner is often
+      # reassigned to the same region. Seen on astubbs/parallel-consumer#394 and
+      # astubbs/parallel-consumer#395 - Unit AND Integration both red at
+      # `unpack (unpack-kafka-streams-sources)` with ZERO tests run, so the lane called "Unit Tests"
+      # names a subsystem that never got as far as compiling.
+      #
+      # Warmed by COORDINATE rather than by building the module. Building it warms the jars too, and
+      # is what the Connect spike branch does (`-pl parallel-consumer-connect -am -DskipTests
+      # package`) - but it costs a full build of core inside this job, and that branch records why it
+      # cannot be shortened to a cheap phase-only walk: the module's test-scope resolution wants
+      # core's tests-classifier jar, which does not exist until core reaches `package`. Two jars are
+      # the whole gap here and `kafka.version` is a real root-pom property, so naming them is both
+      # cheaper and no more prone to rot.
+      #
+      # Verified locally against an empty local repository: the two `get`s fetch exactly the two jars
+      # the assertions name, after which `-pl parallel-consumer-streams -am generate-test-sources`
+      # runs OFFLINE from a cleaned target and both unpack executions succeed - so what this warms is
+      # sufficient for the artifactItems resolve, not merely adjacent to it.
+      #
+      # `kafka.version` is the default every live lane builds at. The one job that overrides it,
+      # `test-kafka-compat`, is `if: false`; re-enabling it puts its Kafka version back outside this
+      # warm. (astubbs#255)
+      - name: Warm the Kafka sources jars the streams module unpacks
+        run: |
+          set -euo pipefail
+          evaluate() {
+            ./mvnw -q --batch-mode -N \
+              -Dexpression="$1" -DforceStdout \
+              org.apache.maven.plugins:maven-help-plugin:3.5.2:evaluate 2>/dev/null | tail -1
+          }
+          # A renamed property makes help:evaluate print "null object or invalid expression", which
+          # dependency:get would happily turn into a warm of the wrong coordinate - and warming
+          # nothing is indistinguishable from a cache hit until it fails a job downstream. Require a
+          # version-shaped answer rather than merely a non-empty one.
+          require_version() {
+            case "$2" in
+              [0-9]*) : ;;
+              *) echo "::error::$1 did not resolve to a version from the root pom (got '$2') - the Kafka sources cache warm would be a silent no-op"; exit 1 ;;
+            esac
+          }
+          kafka_version=$(evaluate kafka.version)
+          require_version kafka.version "$kafka_version"
+          for classifier in sources test-sources; do
+            ./mvnw --batch-mode -ntp \
+              org.apache.maven.plugins:maven-dependency-plugin:3.11.0:get \
+              -Dartifact="org.apache.kafka:kafka-streams:${kafka_version}:jar:${classifier}"
+            # Assert the jar is on disk: the exit code says the goal ran, only the file says the
+            # cache this job exists to populate actually got populated.
+            test -s ~/.m2/repository/org/apache/kafka/kafka-streams/"${kafka_version}"/kafka-streams-"${kafka_version}"-"${classifier}".jar
+          done
       - name: Save Maven cache (rotating key)
         if: success()
         uses: actions/cache/save@v4

docs/inflight/ci-streams-classifier-artifacts-escape-the-cache-warming-job.md

@@ -1,89 +1,69 @@
-# The streams module's `sources`/`test-sources` artifacts are not pre-warmed, so every run of it fetches from Central live
+# The streams module's `sources` jars are warmed HERE, and nowhere else yet
 
 <!-- inflight-type: bug -->
 <!-- inflight-impact: misdirection -->
 
-<!-- post-merge: checked-begin -->
-`parallel-consumer-streams` (astubbs#255). First seen as a red Unit lane on the task-lifecycle rung,
-astubbs/parallel-consumer#394, which had **nothing to do with the tests** on it. Seen again, with the
-same signature, on the error-surfacing rung stacked above it, astubbs/parallel-consumer#395 - two
-independent branches, which is what makes it master-state rather than one PR's problem.
-<!-- post-merge: checked-end -->
-
-## What it looks like
-
-The Unit lane goes red with the streams module `FAILURE` and every other module `SUCCESS`, at
-`maven-dependency-plugin:unpack (unpack-kafka-streams-sources)`:
-
-```
-Could not transfer artifact org.apache.kafka:kafka-streams:jar:sources:3.9.2
-from/to central (https://repo1.maven.org/maven2/): Read timed out
-```
-
-**No test ran.** The module dies in `generate-sources`, so a reader looking for a failing test finds
-none, and the lane's name says "Unit Tests". That is the misdirection: the signal names the wrong
-subsystem.
-
-## Why the cache-warming job does not cover it
-
-`prepare-deps` in `.github/workflows/maven.yml` warms the cache with `dependency:go-offline`, which
-resolves the **declared dependency graph**. This module additionally fetches artifacts by explicit
-`artifactItems` in two `dependency:unpack` executions - Apache Kafka's `sources` and `test-sources`
-classifiers - and those are not part of that graph. So they are downloaded live from Maven Central on
-every run of this module, in exactly the phase where the region-dependent timeout class already
-documented in
+`parallel-consumer-streams` (astubbs#255) fetches Kafka's `sources` and `test-sources` classifier
+jars through `dependency:unpack` `<artifactItems>`, which `dependency:go-offline` does not resolve -
+so they were fetched live from Maven Central inside `generate-sources` on every lane, in exactly the
+phase where the region-dependent read timeout in
 [`docs/solutions/build-errors/maven-central-timeout-azure-west-regions-2026-04-21.md`](../solutions/build-errors/maven-central-timeout-azure-west-regions-2026-04-21.md)
-bites.
-
-That write-up's own conclusion is the relevant one: **re-running does not reliably help**, because the
-runner is often reassigned to the same region, and the fix that worked was pre-warming so nothing is
-fetched from Central during the build. This module is the one place that was left outside the fix.
+bites. It presented as Unit **and** Integration red at `unpack (unpack-kafka-streams-sources)` with
+zero tests run, so the lane called "Unit Tests" named a subsystem that never reached compilation.
 
-**That prediction has now been tested, and "not reliably" is exactly the right strength - read it as
-written rather than as "never".** On the error-surfacing rung it fired on two consecutive runs: the
-second reproduced the identical failure at the same execution on the same artifact, and took the
-**Integration lane down with the Unit lane** on the same run. A third run then passed every lane. So:
-
-- **The exposure is not "one lane occasionally"** - it is *every lane that builds this module*, and
-  they fail together, because they share the runner's route to Central.
-- **It is per-run, not per-branch.** A run either has the problem or does not, so a re-run is a coin
-  flip rather than a fix, and a green run is luck that says nothing about the next one.
-- **A green lane is therefore not evidence this is closed.** It is closed when the unpack executions
-  stop reaching Central at all - see *Delete when* below.
-
-**And it has nothing to do with the diff, which a control arm now settles rather than argues.** The
-run after the green one carried a **markdown-only commit** - no Java, no pom, no workflow - and
-reproduced the identical failure on both lanes. There is no reading of that in which the change under
-review is implicated. If you are looking at a red Unit lane on this module and wondering what you
-broke, this paragraph is the answer.
-
-## Candidate fix
+<!-- post-merge: checked-begin -->
+**Fixed by astubbs/parallel-consumer#379**, which added the `Warm the Kafka sources jars the streams
+module unpacks` step to `prepare-deps` in `.github/workflows/maven.yml` - so wherever that step is
+present, this is closed. **That step's own comment is the durable owner** of why go-offline
+misses an artifactItem, why the warm names coordinates rather than building the module, and what the
+two guards are for; this note does not restate it.
+<!-- post-merge: checked-end -->
 
-Add the classifier artifacts to what `prepare-deps` pulls, so the warm cache actually contains
-everything a build needs. `dependency:go-offline` will not do it on its own; the honest options are
-running the module's `generate-sources` phase in the warming job, or a `dependency:get` per artifact
-item. Whichever is chosen, the check that it worked is that the streams module's `unpack` executions
-log a cache hit rather than a `Downloading from central` line.
+## What the sightings established, for any branch still exposed
 
-## Until it is fixed, what a blocked PR should do
+<!-- post-merge: checked-begin -->
+The defect was observed on two independent rungs of this stack -
+astubbs/parallel-consumer#394 and astubbs/parallel-consumer#395 - which is what made it master-state
+rather than one PR's problem. Three findings from those sightings outlive the fix, because they
+describe what an *unwarmed* branch is still living with:
+
+- **It is every lane that builds this module, not one lane occasionally.** Unit and Integration went
+  red together on the same run, because they share the runner's route to Central.
+- **It is per-run, not per-branch, so a re-run is a coin flip and not a fix.** The referenced
+  write-up's "re-running does not reliably help" proved to be exactly the right strength: two
+  consecutive runs reproduced the identical failure at the same execution on the same artifact, and a
+  third then passed every lane. A green run is luck, and says nothing about the next one.
+- **It has nothing to do with the diff, and a control arm settles that rather than arguing it.** A
+  **markdown-only commit** - no Java, no pom, no workflow - reproduced the failure on both lanes
+  immediately after a green run. There is no reading of that in which the change under review is
+  implicated.
+
+If you are on an unwarmed branch looking at a red Unit lane on this module and wondering what you
+broke: the answer is the paragraph above, and the fix is to merge astubbs/parallel-consumer#379
+forward rather than to re-run.
+<!-- post-merge: checked-end -->
 
-**Read the log before concluding anything.** The lane is named "Unit Tests" and no test ran, so the
-first job is to recognise it: the module dies at `unpack-kafka-streams-sources`, every other module
-succeeds, and the message names Central and the `sources` classifier.
+## What is still open
 
-Then **say in the PR that the red lane is this, name this note, and put the module's own local lane
-up as the evidence the code is healthy** - the whole `test` phase and the integration lane, counts
-read out of the report directories. A local run has the artifacts in `~/.m2` already, which is why it
-never reproduces there, and why "green locally" is a real signal about the code and none at all about
-this.
+<!-- post-merge: checked-begin -->
+**Every branch carrying `parallel-consumer-streams` without that step still has the unwarmed
+workflow**, and stays exposed until it merges astubbs/parallel-consumer#379 forward. The candidates
+are
+`for r in $(git for-each-ref --format='%(refname:short)' refs/remotes/origin); do git cat-file -e "$r:parallel-consumer-streams/pom.xml" 2>/dev/null && echo "$r"; done`,
+minus those that already contain the step - `git grep -l 'Warm the Kafka sources jars' <ref> --
+.github/workflows/maven.yml`.
+
+**A branch carrying the pre-fix copy of this file collides add/add on this path** when it merges
+astubbs/parallel-consumer#379 forward: **take the version that names the step**, not the one that
+calls the fix a candidate. The two rungs that hit it, `feats/ks-streams-task-lifecycle` and
+`feats/ks-streams-error-surfacing`, resolved it that way and are no longer exposed; the instruction
+stands for any rung above them that has not yet merged forward.
+<!-- post-merge: checked-end -->
 
-**Re-running is legitimate here and is not a retry-into-green** - it is not a flaky test being
-papered over, it is an artifact download that either reached Central or did not, and no test outcome
-is being hidden. What it is not is a fix: it is a coin flip, it can take several attempts, and the
-green it eventually produces is not evidence the problem is gone. So re-run if you need the lane, and
-**do not let the green talk you out of leaving the record behind**.
+**`test-kafka-compat` is the one job whose Kafka version falls outside the warm.** It is `if: false`
+today; re-enabling it re-opens this for that lane only.
 
 ## Delete when
 
-A streams-module CI run shows the unpack executions resolving from the warmed cache rather than
-downloading from Central.
+No open branch builds the streams module without that step - at which point nothing here is both
+true and unowned by the workflow comment.

docs/inflight/test-streamthreadtest-invalid-timestamps-flake.md

@@ -36,6 +36,13 @@ it costs every branch in the chain a red build and a re-run, and where the tempt
 green is strongest.
 <!-- post-merge: checked-end -->
 
+**Sighted twice more on the example rung's CI** (astubbs/parallel-consumer#391): across four
+attempts of its Unit lane, the sequence was flake, Maven Central timeout, green, flake - two
+failures in the three attempts that actually ran the suite, at the full run count the oracle
+produces on this stack. Consistent with the forest's originally measured rate rather than below it.
+The example agent deliberately stopped re-running to chase green, which is the habit this note
+exists to prevent.
+
 ## Why this matters more than one flaky test
 
 **Every agent working on this module is told that "Kafka's own suites, zero failures with the seam