feat(server,deps,ci): update dependencies, fix CI scoping, exit on shard panic - #4033
Open
hubcio wants to merge 5 commits into
Open
feat(server,deps,ci): update dependencies, fix CI scoping, exit on shard panic#4033hubcio wants to merge 5 commits into
hubcio wants to merge 5 commits into
Conversation
hubcio
force-pushed
the
update-all-deps-workspace
branch
from
September 2, 2026 06:39
0109e5b to
b0e1065
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #4033 +/- ##
============================================
+ Coverage 85.11% 85.12% +0.01%
Complexity 1402 1402
============================================
Files 1236 1236
Lines 181465 181439 -26
Branches 147760 147732 -28
============================================
+ Hits 154445 154455 +10
+ Misses 22966 22934 -32
+ Partials 4054 4050 -4
🚀 New features to boost your workflow:
|
hubcio
force-pushed
the
update-all-deps-workspace
branch
from
September 2, 2026 06:58
b0e1065 to
76431ab
Compare
spetz
previously approved these changes
Sep 2, 2026
slbotbm
previously approved these changes
Sep 2, 2026
hubcio
force-pushed
the
update-all-deps-workspace
branch
from
September 2, 2026 09:09
76431ab to
5a31ee5
Compare
hubcio
changed the title
hubcio
changed the title
Bump every workspace crate to its latest release and carry the majors through their API changes: comfy-table 8 renames load_preset to load_style, apache-avro 0.22 replaces the free datum functions with GenericDatumReader/Writer builders and makes the JSON to Avro conversion fallible, argon2 0.6 takes raw salt bytes and the encoded hash string directly instead of SaltString/PasswordHash, and kafka-protocol 0.18 adds delete_horizon to Record. Deliberately held back: arrow/parquet stay on 58 because iceberg 0.10.1 and deltalake 0.32.4 still require it, testcontainers stays on 0.27 for testcontainers-modules 0.15, and ext-php-rs keeps the =0.15.14 pin from #3411. The compio exact pins from #3192 had no recorded reason and are lifted, moving compio to 0.19.2 and compio-quic to 0.8.2. Every member now inherits its dependencies from the workspace: shard, simulator, integration's Doris sink and the Kafka gateway plus its tool declared path or version entries of their own. The kafka-protocol entry drops default features, so the tool no longer links the compression codecs it never uses. Stale manifest notes are gone too: syn is on 3 with darling 0.24, getrandom is an argon2 0.6 default, and the dev-profile override now targets chacha20 because rand 0.10 dropped rand_chacha. The foreign SDKs move alongside: rules_rust 0.74.0 with host tools on 1.98.0 to match rust-toolchain.toml, phpunit ^12.5 (13.x needs PHP 8.4, CI runs 8.3) and a maturin 1.15 floor. The Python pylock.toml had been stale since #3323 and is regenerated; the bdd and examples uv.lock files are re-locked because they embed the SDK's extras specifiers.
compio-runtime 0.12.6 runs block_on inside catch_unwind and moves the future through several extra frames. Unoptimized builds keep a copy in each, so a #[compio::test] now needs roughly fifteen times its future size in stack instead of five. The dead-incarnation rebuild test held five 32 KiB IggyPartition values across awaits: a 160 KiB future that needed 2.7 MiB against the 2 MiB test thread default and aborted every CI retry with "has overflowed its stack". Boxing the four rebuilt partitions brings it back under 900 KiB, in line with the other purge tests. Release builds elide the copies, so iggy-server is unaffected.
Unpinned cargo-rail moved to 0.24 on 2026-08-28 and dropped the `-f json` flag the DAG scoping step passes. The step only warns "Could not compute affected crates", so every PR since has built and run the full workspace on all three shards. 0.25 also rejects the [change-detection] table in rail.toml, so moving forward means rewriting the config and both plan consumers; pin 0.23.0 in the pre-merge action and in post-merge and bump on purpose. On those full-workspace lanes the Kafka fixture gate diffed origin/master...HEAD. actions/checkout is depth 1 and the base fetch is depth 1, so there is never a merge base and the fail-safe built the fixtures on every shard. Read the merge commit's first parent from the raw object, fetch that one commit and diff against it; a HEAD without two parents keeps the fail-safe.
compio's `spawn` catches task panics, and nothing observed them while the server ran: the pump's handle is awaited only at shutdown, and `track_background` reaps listener and connection handles without reading their result. A panicking pump left its partitions stalled until an operator noticed. A panicking listener or connection task left the server running without it and the eventual Ctrl-C exited 0. The thread-level path (`ShutdownOnDrop`) covered only panics that unwound out of `run_shard_thread`, and no path logged a backtrace. Install a process-wide panic hook in `bootstrap` before the first shard thread spawns. It logs the panic with thread, location and a forced backtrace through tracing, records the first one, and flips the shared shutdown flag so every shard drains the same way a thread-level failure already does. `join_all` fails the exit with `ServerError::Panicked` when every thread exited `Ok` but a panic was recorded, so an orchestrator never reads a panic as a clean stop. Graceful drain rather than abort keeps the pump's final journal flush on the non-pump panic paths. A pump panic still surfaces as `ShardPumpDied` through the existing drain verdict.
The doc job runs `cargo doc --no-deps --all-features` under `RUSTDOCFLAGS=-D warnings`, and rustdoc rejects a public struct's docs linking to a `pub(in crate::boot)` fn.
hubcio
force-pushed
the
update-all-deps-workspace
branch
from
September 2, 2026 09:47
d89ce27 to
0f06eb3
Compare
mmodzelewski
approved these changes
Sep 2, 2026
spetz
approved these changes
Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bump every workspace crate to its latest release and adapt to the
majors (comfy-table 8, apache-avro 0.22, argon2 0.6, kafka-protocol
0.18). Held back by consumer pins: arrow/parquet 58, testcontainers
0.27, ext-php-rs =0.15.14 (#3411). Foreign SDKs move alongside;
stale pylock.toml and bdd/examples uv.lock regenerated.
compio-runtime 0.12.6 runs block_on inside catch_unwind and debug
builds copy the future into every wrapper frame, so a purge test
holding five IggyPartition values across awaits overflowed the 2 MiB
test thread. Its rebuilt partitions are now boxed; the server is
unaffected.
CI: pin cargo-rail 0.23.0 (0.24 dropped -f json; every PR since
2026-08-28 silently ran the full workspace) and diff the Kafka fixture
gate against the merge commit's first parent; origin/master...HEAD has
no merge base on a depth-1 checkout.
Also carries the shard panic hook: log a backtrace, drain every shard
and exit with ServerError::Panicked instead of running on.