bench: add Skiff format throughput coverage (#51) #95
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: CI | |
| on: | |
| push: | |
| branches: [main] | |
| pull_request: | |
| workflow_dispatch: | |
| env: | |
| CARGO_TERM_COLOR: always | |
| RUST_BACKTRACE: 1 | |
| jobs: | |
| go-skiff-reference: | |
| name: pinned Go Skiff reference | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v5 | |
| - uses: actions/setup-go@v6 | |
| with: | |
| go-version: "1.25.x" | |
| - name: Go SDK v0.0.33 Skiff vectors | |
| working-directory: tests/skiff-go-interop | |
| run: go test ./... | |
| check: | |
| name: fmt + clippy + test | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v5 | |
| # rust-toolchain.toml pins the channel, components and targets. | |
| - name: Install toolchain | |
| run: rustup show active-toolchain || rustup toolchain install | |
| - uses: Swatinem/rust-cache@v2 | |
| - name: cargo fmt --check | |
| run: cargo fmt --all -- --check | |
| - name: cargo clippy | |
| run: cargo clippy --workspace --all-targets -- -D warnings | |
| # The client without TLS is what a worker binary links against, and the | |
| # `https://` refusal only exists in that build. Nothing else compiles it. | |
| - name: cargo clippy (client without TLS) | |
| run: cargo clippy -p ytsaurus-client --no-default-features --all-targets -- -D warnings | |
| # `derive` and `tracing` are both off by default, so the workspace build | |
| # compiles neither the macro against the client's own types nor the spans | |
| # and the `tracing`-shaped retry reporting. Clippy as well as the tests: | |
| # the two spellings of `observe` are separate code, and only one of them | |
| # is linted above. | |
| - name: cargo clippy (client with every feature) | |
| run: cargo clippy -p ytsaurus-client --all-features --all-targets -- -D warnings | |
| - name: cargo test (client with every feature) | |
| run: cargo test -p ytsaurus-client --all-features | |
| # The workers are examples of `ytsaurus-job`, and the end-to-end tests | |
| # below exec them. `--all-targets` does not build a runnable one — there | |
| # `--examples` means *test* the examples, so each is compiled as a libtest | |
| # harness and no plain binary is produced — and `tests/common::example` | |
| # answers that by building what it needs. Doing it here as well is not | |
| # redundant: it builds the nine once, before any test runs, instead of | |
| # leaving several test binaries to discover the gap at the same moment and | |
| # queue behind each other on cargo's lock. | |
| - name: Build the worker examples the e2e tests run | |
| run: cargo build -p ytsaurus-job --examples | |
| - name: cargo test | |
| run: cargo test --workspace --all-targets | |
| # `--all-targets` skips doctests, and the mapper example in the | |
| # ytsaurus-job docs must keep compiling. | |
| - name: cargo test --doc | |
| run: cargo test --workspace --doc | |
| # `--all-targets` also skips `#[ignore]`d tests, so the 2 GB streaming | |
| # test needs its own step or it never runs. It is the only guard against | |
| # the reader silently starting to accumulate input: ~10 s in debug, and | |
| # it asserts peak RSS stays far below the budget. | |
| - name: cargo test (2 GB streaming memory test) | |
| run: cargo test -p ytsaurus-job --test memory_tests -- --ignored --nocapture | |
| musl: | |
| name: static musl worker build | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v5 | |
| - name: Install toolchain | |
| run: rustup show active-toolchain || rustup toolchain install | |
| - uses: Swatinem/rust-cache@v2 | |
| # `tls` and `tracing` are both off in a worker build, and both would | |
| # break or bloat it: `rustls` reaches `ring`, which wants a C | |
| # cross-compiler, and a worker should carry only what it runs on. Neither | |
| # is off by accident — `ytsaurus-job` takes the client as a | |
| # `default-features = false` dev-dependency, for the `selfrun` example — | |
| # so this asserts the invariant rather than trusting it. | |
| # | |
| # Read **with** dev-dependencies, which is the whole point: cargo compiles | |
| # a package's dev-dependencies whenever it builds that package's examples, | |
| # and the workers are examples. That is also why criterion is pinned | |
| # below 0.8: 0.8 reaches `alloca`, whose build script wants the C | |
| # cross-compiler this build is meant not to need. `rustls-platform-verifier` is the third name for the same | |
| # reason: the `platform-verifier` feature is gated on `tls`, and this is | |
| # what says so out loud. | |
| # | |
| # Listed once and searched, rather than three `cargo tree -i` calls whose | |
| # *failure* was the passing signal. `-i` exits 101 both when the crate is | |
| # absent and when cargo itself could not run — a typo'd `-p`, a manifest | |
| # or lockfile error, an unreachable registry — so reading the exit code | |
| # turned every one of those into a silent pass. The message does not | |
| # separate them either: `-i` is resolved before `-p`, so a misspelled | |
| # package prints the same "did not match any packages" as an absent | |
| # crate. One invocation that must succeed puts all of that on the failing | |
| # side of `set -e` and leaves a plain membership test. | |
| - name: Assert the worker graph has no tracing and no TLS | |
| run: | | |
| set -euo pipefail | |
| graph=$(cargo tree -p ytsaurus-job \ | |
| --target x86_64-unknown-linux-musl \ | |
| --prefix none --no-dedupe | awk 'NF {print $1}' | sort -u) | |
| # A graph that lost the client is a graph this step is not reading, | |
| # and every absence below would be vacuously true. | |
| echo "$graph" | grep -qx ytsaurus-client || { | |
| echo "ERROR: the worker graph does not contain ytsaurus-client" | |
| echo "$graph"; exit 1; } | |
| for crate in tracing rustls ring rustls-platform-verifier; do | |
| if echo "$graph" | grep -qx "$crate"; then | |
| echo "ERROR: $crate reached the worker build" | |
| cargo tree -p ytsaurus-job \ | |
| --target x86_64-unknown-linux-musl -i "$crate" | |
| exit 1 | |
| fi | |
| echo "OK: no $crate in the worker graph" | |
| done | |
| # Through the script rather than a bare cargo call: the script is what | |
| # stages each example from `<profile>/examples/<name>` into | |
| # `<profile>/<name>`, which is the layout the assertion below and every | |
| # document in the repository expect. Running it here also means the thing | |
| # contributors run is the thing CI exercises. | |
| - name: Build worker binaries | |
| run: ./scripts/build-worker.sh | |
| - name: Assert binaries are statically linked | |
| run: | | |
| set -euo pipefail | |
| shopt -s nullglob | |
| bins=(target/x86_64-unknown-linux-musl/release-worker/*) | |
| found=0 | |
| for f in "${bins[@]}"; do | |
| [ -f "$f" ] && [ -x "$f" ] || continue | |
| case "$f" in *.d|*.rlib) continue;; esac | |
| echo "--- $f" | |
| file "$f" | |
| # rustc emits static-pie for musl; both spellings mean "no libc.so". | |
| file "$f" | grep -Eq 'statically linked|static-pie linked' | |
| # A dynamically linked binary would list shared objects here. | |
| ! ldd "$f" 2>&1 | grep -q '=> /' || { | |
| echo "ERROR: $f has dynamic dependencies"; exit 1; } | |
| found=$((found + 1)) | |
| done | |
| if [ "$found" -eq 0 ]; then | |
| echo "ERROR: no worker binaries were produced"; exit 1 | |
| fi | |
| echo "OK: $found statically linked worker binaries" |