Commit ec432b4
committed
ci(streams) #255: warm the Kafka sources jars the streams module unpacks
The streams module gets Kafka's `sources` and `test-sources` classifier jars
through two `dependency:unpack` executions that name them as `<artifactItems>`.
`prepare-deps` warms the cache with `dependency:go-offline`, which resolves the
declared dependency graph - and an artifactItem is not in that graph, nor is
either jar declared anywhere else. So both stayed cold, and every lane that
built this module fetched them from Maven Central inside `generate-sources`.
That put them on the CDN route lottery recorded in
docs/solutions/build-errors/maven-central-timeout-azure-west-regions-2026-04-21.md:
an exactly-240s read timeout, and re-running does not reliably help because the
runner is often reassigned to the same region. It struck
#394 and #395 - Unit and
Integration both red at `unpack (unpack-kafka-streams-sources)` with zero tests
run, so the lane named "Unit Tests" pointed at a subsystem that never reached
compilation.
Same plugin-resolves-for-itself class the proxy stack already warms three times
over (protoc, scala-maven-plugin, the SpotBugs detector plugins), reached from a
new direction: here the plugin resolving for itself is maven-dependency-plugin.
The step follows that established pattern - version derived from the pom
property rather than a literal, a non-version help:evaluate answer is a hard
error naming the property, and the jars are asserted on disk afterwards.
Rejected alternative: warming by building the module, which is what the Connect
spike branch does. It works, but costs a full build of core inside this job, and
that branch records why it cannot be shortened to a cheap phase-only walk - the
module's test-scope resolution wants core's tests-classifier jar, which does not
exist until core reaches `package`. Two jars are the whole gap and
`kafka.version` is a real root-pom property, so naming them is cheaper and no
more prone to rot.
Verified by extracting the step's script from the workflow and running it:
- Empty local repository, script otherwise verbatim: exit 0, and both jars
arrive at exactly the paths the assertions name, byte-identical to the
populated repository's copies.
- Property renamed to `kafka.verzion`, one term changed and nothing else:
exit 1, `::error::kafka.verzion did not resolve to a version from the root pom
(got 'null object or invalid expression')`, and nothing fetched. The guard
fires and names the property.
- On-disk assertion is independently load-bearing: a run where the `get` goal
succeeded but the asserted path was elsewhere exited 1 on the `test -s`, which
is the case the exit code alone would have called a pass.
- Sufficiency: after the warm, `-pl parallel-consumer-streams -am
generate-test-sources` runs OFFLINE from a cleaned target and both unpack
executions succeed, producing the unpacked sources. Offline makes any Central
fetch a hard failure, so this shows the warmed jars satisfy the artifactItems
resolve rather than merely sitting near it.
`kafka.version` is the default every live lane builds at. The one job that
overrides it, `test-kafka-compat`, is `if: false`; the comment says so, so
re-enabling it comes with the note that its Kafka version falls outside this
warm.1 parent 3fa6b6f commit ec432b4
1 file changed
Lines changed: 58 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
58 | 116 | | |
59 | 117 | | |
60 | 118 | | |
| |||
0 commit comments