diff --git a/.github/workflows/fleet-watchdog.yml b/.github/workflows/fleet-watchdog.yml new file mode 100644 index 0000000..7d24163 --- /dev/null +++ b/.github/workflows/fleet-watchdog.yml @@ -0,0 +1,36 @@ +# Alerts (via a repo issue) when the Railway bench fleet is left running +# between campaigns. Railway bills per-minute for allocated resources; +# teardown is the mandatory final step of every run, and this is the net +# under it. See scripts/fleet-watchdog.mjs. +# +# Setup (one-time, repo admin): +# 1. Create a Railway ACCOUNT token at railway.com/account/tokens +# (CLI login tokens expire; account tokens persist). +# 2. Add it as the RAILWAY_TOKEN repository secret. +# PROJECT_ID / ENVIRONMENT_ID below match fleet-manifest.json. + +name: fleet-watchdog + +on: + schedule: + - cron: "17 */6 * * *" + workflow_dispatch: + +permissions: + issues: write + +jobs: + check: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: 22 + - run: node scripts/fleet-watchdog.mjs + env: + RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }} + PROJECT_ID: fd842a43-8d78-48c0-879f-4b5311c8c004 + ENVIRONMENT_ID: 284c9b0a-57cb-4281-a4bd-868473a94a47 + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + GITHUB_REPOSITORY: ${{ github.repository }} diff --git a/.gitignore b/.gitignore index dafc8dd..9ddd298 100644 --- a/.gitignore +++ b/.gitignore @@ -2,11 +2,13 @@ node_modules/ dist/ *.log -# Bench result outputs go in backend/results/. The directory is tracked -# via .gitkeep so scripts can write into it on a fresh clone; the actual -# result files (CSV/JSON) are ignored. +# Bench result outputs go in backend/results/. Raw per-run dumps (timestamped +# CSV/JSON) are ignored, but the curated, published result files that the +# README and docs cite are kept in the repo. backend/results/* !backend/results/.gitkeep +!backend/results/rails-*.json +!backend/results/socketioxide-*.json # AnyCable Pro binaries are licensed and MUST NEVER be committed to this # public repo. These patterns match the asset names from the private diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..a0c5d04 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,18 @@ +# Working on this benchmark repo + +Read `docs/methodology.md` first; `docs/railway-ops.md` for fleet mechanics. When working from the anycable-web repo, the full playbook lives in `.claude/skills/benchmarking/` there. + +## Rules that are never optional + +1. **Run `npm run bench:preflight` (from `backend/`) before any paid run window.** It verifies runner image freshness, secrets parity across shards, target env, and deploy churn. `npm run bench:fleet` shows what is live and billing. +2. **The load generator is a suspect in every result.** Keep ~250 cables per runner for latency/jitter, ~1 runner per 10K connections for capacity. Multi-shard drivers self-flag validity problems (delivery over 100%, uniform shard ceilings, negative skew floor, elapsed overrun); a fatal flag means fix and rerun, never publish. +3. **Never ship a load-generator-limited number.** Identical per-shard ceilings with a healthy server = the fleet's wall, not the server's. +4. **Same-window comparisons only.** Every row of one table comes from one continuous window, zero deploy churn during runs; re-run outliers in isolation. +5. **Fairness before running:** worker counts from boot logs (stock puma.rb ignores `WEB_CONCURRENCY` without a `workers` directive), equal box sizes set explicitly, native client per adapter for reconnect-dependent tests, same enforced outage for every client. +6. **Teardown ends every window**: `railway down` or `deploymentRemove` per service (never `deploymentStop`; limits changes alone keep billing), verify domains return 404, restore any temporarily altered config. The fleet-watchdog Action opens an issue if something is left running. +7. **Baselines live in `src/bench/tests-manifest.ts`**; update them in the same commit as the change that moved them. New query params must be registered in `src/bench-runner/known-params.ts` or every driver sending them fails fast (by design). +8. After a bench-runner code change, `railway up` every shard in use and verify deployment `createdAt` postdates the commit; `railway redeploy` reuses the old image and git push rebuilds nothing. + +## Writing style (README, docs, PR bodies) + +Succinct, alive, no throat-clearing. No em dashes. State the positive directly (no "not X, but Y" constructions). Keep caveats attached to the numbers they qualify; a public table must never look cleaner than the underlying data. diff --git a/README.md b/README.md index 3c938ba..8c87cd9 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,8 @@ Setups under test: default Socket.io, Socket.io + Connection State Recovery, uWe Sixth target, [socketioxide](https://github.com/totodore/socketioxide) (Rust Socket.io server), benchmarked head-to-head with AnyCable on the same Railway hardware. Results in [its own section below](#socketioxide-rust-socketio); deep dive in [`docs/socketioxide-comparison.md`](./docs/socketioxide-comparison.md). +The same harness also drives the **Rails** comparison (Action Cable vs Solid Cable vs Async::Cable vs AnyCable) behind [anycable.io/compare/rails-actioncable](https://anycable.io/compare/rails-actioncable): one Rails app, four adapters, same box. Results in [its own section below](#rails-action-cable--solid-cable--asynccable--anycable); deep dive in [`docs/rails-comparison.md`](./docs/rails-comparison.md). + Methodology, traps, and the bugs we caught in our own setup: [`docs/methodology.md`](./docs/methodology.md). Below: the numbers and how to rerun them. ## Headlines @@ -110,6 +112,45 @@ Three knobs that shape the numbers. Full reasoning in [`docs/methodology.md`](./ **The takeaway.** Rust fixes Socket.io's capacity ceiling, the single-event-loop wall that caps Node around 120K. It leaves two things untouched: at-most-once delivery (no replay protocol) and deploy fragility (the WS layer still dies with its app). Both live in the protocol and the topology, so swapping the language to Rust leaves them intact, and socketioxide collapses under jitter and deploy storms at scale the same way Node Socket.io does. AnyCable holds 100% on both because the WS layer is a separate process with replay. Full numbers and the deploy story: [`docs/socketioxide-comparison.md`](./docs/socketioxide-comparison.md). +## Rails (Action Cable / Solid Cable / Async::Cable / AnyCable) + +The repo behind [anycable.io/compare/rails-actioncable](https://anycable.io/compare/rails-actioncable). Four WebSocket adapters for the **same Rails 8.1 app**, same Railway box, same shared-tenant window: Action Cable on Redis (Puma), Solid Cable on the database (Puma), Async::Cable on Falcon (`socketry/async-cable`, fiber reactor), and AnyCable in RPC mode (Rails gRPC backend + `anycable-go` gateway). All four are Action Cable-compatible at the app and client level, so the channels and Turbo Streams are identical; only `config/cable.yml`, the runtime, and the process topology change. The Puma targets are in `cable-bench/`, the Falcon target in `cable-bench-falcon/`; the AnyCable JS driver serves all four over the existing `bench-*-anycable` endpoints, parameterized by `cableUrl` / `broadcastUrl` / `channel` / `acProtocol`. + +The one wire difference: Action Cable, Solid Cable and Async::Cable speak the base `actioncable-v1-json` protocol (at-most-once, no resume); AnyCable speaks the extended `actioncable-v1-ext-json` protocol (per-stream history, resume on reconnect). Switching Puma for Falcon (Async::Cable) changes the runtime, not the guarantees. + +All numbers below are **sharded** (13 drivers, ~250 to 770 cables each). A single bench-runner holding thousands of cables saturates its own event loop and produces wrong numbers in both directions; see [Running the latency test](#running-the-latency-and-jitter-test-shard-it). + +**Roundtrip latency (steady network, 100% delivery).** AnyCable leads at every scale: fan-out runs in Go, off the Ruby process. + +| Adapter | 1K p50 / p99 | 5K p50 / p99 | +| --- | --- | --- | +| Solid Cable | 62 / 119 ms | 74 / 164 ms | +| Async::Cable (Falcon) | 11 / 80 ms | 20 / 71 ms | +| Action Cable (Puma) | 9 / 47 ms | 13 / 57 ms | +| AnyCable | **4 / 23 ms** | **7 / 31 ms** | + +**Delivery under jitter (5K, ~2 s drops).** The base protocol has no resume, so the three in-process adapters lose the same ~22%; AnyCable replays per-stream history. + +| Adapter | Delivery | +| --- | --- | +| Solid Cable | **78.1%** | +| Action Cable | **78.1%** | +| Async::Cable (Falcon) | **78.1%** | +| AnyCable | **99.9%** | + +**Capacity: 10K under load + idle-to-break.** All four hold 10K at 100% delivery; AnyCable keeps the tightest tail (p99 31 ms vs 84 ms Action Cable, 112 ms Async::Cable, 200 ms Solid Cable). Pushed to failure on identical 32 GB boxes (default 8-worker config): Puma Action Cable and Solid Cable wall at ~52K on a file-descriptor ceiling (~2.5 GB RAM, not memory-bound); Async::Cable on Falcon runs out of memory at ~97K (~290 KB/conn); AnyCable held **600K with zero failures** across a 50-runner fleet (~47 KB/conn, ~27 GB, ~84% of the box), not driven to failure: the load fleet maxed out, not the server, so treat 600K as a floor. Finding a gateway's true ceiling takes ~1 load driver per 10K connections (a single Node driver tops out near 10K cables), so capacity-to-break runs scale the driver count to the target. Raw data in [`backend/results/rails-capacity-break-2026-06-28.json`](./backend/results/rails-capacity-break-2026-06-28.json). + +**Deploy survival (avalanche, 5K, real app redeploy).** In-process drops every connection, Puma and Falcon alike, and stays down ~7.5 to 8 s before ~96% reconnect; AnyCable's gateway never sees the deploy, so 0 s of downtime. "Down for" is how long connections stayed dropped before the reconnect storm settled. + +| Adapter | Dropped | Down for | Reconnected | +| --- | --- | --- | --- | +| Action Cable | all 5,000 | 7.5 s | 96.3% (187 still out at cutoff) | +| Solid Cable | all 5,000 | 7.6 s | 95.7% (215 still out at cutoff) | +| Async::Cable (Falcon) | all 5,000 | 8.0 s | 96.4% (179 still out at cutoff) | +| AnyCable | **0** | **0 s** | n/a | + +**The takeaway.** On Rails, AnyCable leads on latency at every scale (7 ms p50 at 5K) and wins the things that decide whether realtime holds up: 100% delivery under jitter where the base protocol drops about a fifth of broadcasts, and connections that survive every app deploy. Capacity ties at everyday sizes (all four hold 10K at 100%) but splits past that: AnyCable held 600K idle connections where the Puma adapters wall at ~52K on a file-descriptor ceiling and Falcon runs out of memory at ~97K. Async::Cable on Falcon is a real alternative runtime to Puma with latency in the same range, but shares the in-process limits (at-most-once, deploy-fragile) and is the most memory-hungry by far. Solid Cable's edge is operational (no Redis), at the cost of a polling-latency floor. Full numbers: [`docs/rails-comparison.md`](./docs/rails-comparison.md). Raw results: [`backend/results/rails-sharded-2026-06-28.json`](./backend/results/rails-sharded-2026-06-28.json). + ## Repository layout ``` @@ -123,7 +164,7 @@ benchmark/ └── backend/ ├── Dockerfile # One image; SERVICE_ENTRY picks the entry point ├── package.json - ├── results/ # CSV/JSON output (gitignored) + ├── results/ # published rails-*/socketioxide-* results tracked; raw run dumps ignored └── src/ ├── publisher.ts # Standalone HTTP publisher (legacy) ├── socketio/server.ts # /_broadcast + /publish-local @@ -314,6 +355,26 @@ INCLUDE_AVALANCHE=1 # add 5 avalanche tests (auto-redeploys server) Full sweep: ~90 minutes wall-clock. +### Running the latency (and jitter) test: shard it + +**Latency and jitter must be sharded. One bench-runner holding thousands of client cables saturates its single Node event loop, and the measured receive latency then reflects the test driver queueing frames, not the server.** This bites hardest for the `actioncable-v1-ext-json` client (AnyCable), which does extra per-message offset/ack work, so an unsharded run makes AnyCable look several times slower than it is. Keep each shard light: **~250 client cables per bench-runner** (the nodejs page uses 250/shard; 1K = 4 shards, 5K = 20 shards, 10K = 40 shards). The coordinator merges the union latency distribution across shards (`mergeJitterResults`), so percentiles stay honest. See [`docs/methodology.md`](./docs/methodology.md#cross-shard-percentile-honesty). + +Set `numShards` + `perShardN` on the spec in `tests-manifest.ts` (e.g. `numShards: 20, perShardN: 250` for 5K), then point `BENCH_RUNNER_URLS` at **at least that many ONLINE runners**. The default pool assumes `bench-runner` + `bench-runner-2..50`; if some are stopped, list the live ones explicitly: + +```bash +cd backend +# Build a URL list from a contiguous range of online runners (here 14..33 = 20 shards): +URLS=$(for i in $(seq 14 33); do printf 'https://bench-runner-%s-production.up.railway.app,' "$i"; done) + +BENCH_RUNNER_TOKEN= \ +BENCH_RUNNER_URLS="${URLS%,}" \ +FILTER=latency \ +OUTPUT_DIR=tmp/latency-run \ + npm run bench:rebaseline +``` + +A spec without `numShards`/`perShardN` runs on a single bench-runner: fine for connection-count or delivery checks, wrong for latency. If a latency p50 jumps super-linearly with subscriber count (e.g. 15 ms at 1K to 225 ms at 5K), suspect an unsharded run before you suspect the server. + **Baselines vs the page numbers.** The page was captured during a noisy Railway shared-tenant window. The `baseline` field in `tests-manifest.ts` is what the same tests deliver on a quieter window: latencies ~50% better, everything else the same. So the page is the cautious "worst seen under shared-infra load" view, and the rebaseline tests against today's quieter floor. A green rebaseline says "we still beat today's floor", which is stricter than the page promises. When we refresh the page, baselines and page numbers move together. Per-run history lives at `tmp/v1.6.14-bench-results/runs/{ISO-ts}/`. To watch each headline number move across runs: diff --git a/backend/package-lock.json b/backend/package-lock.json index ab259a2..549232c 100644 --- a/backend/package-lock.json +++ b/backend/package-lock.json @@ -9,8 +9,10 @@ "version": "1.0.0", "dependencies": { "@anycable/core": "^1.1.6", + "@rails/actioncable": "^8.1.300", "@socket.io/redis-adapter": "^8.3.0", "@socket.io/redis-streams-adapter": "^0.3.1", + "centrifuge": "^5.7.0", "express": "^4.21.0", "ioredis": "^5.10.1", "nats": "^2.29.3", @@ -497,6 +499,69 @@ "node": ">= 10" } }, + "node_modules/@protobufjs/aspromise": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/aspromise/-/aspromise-1.1.2.tgz", + "integrity": "sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/base64": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/base64/-/base64-1.1.2.tgz", + "integrity": "sha512-AZkcAA5vnN/v4PDqKyMR5lx7hZttPDgClv83E//FMNhR2TMcLUhfRUBHCmSl0oi9zMgDDqRUJkSxO3wm85+XLg==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/codegen": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.5.tgz", + "integrity": "sha512-zgXFLzW3Ap33e6d0Wlj4MGIm6Ce8O89n/apUaGNB/jx+hw+ruWEp7EwGUshdLKVRCxZW12fp9r40E1mQrf/34g==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/eventemitter": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/eventemitter/-/eventemitter-1.1.1.tgz", + "integrity": "sha512-vW1GmwMZNnL+gMRaovlh9yZX74kc+TTU3FObkkurpMaRtBfLP3ldjS9KQWlwZgraRE0+dheEEoAxdzcJQ8eXZg==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/fetch": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/fetch/-/fetch-1.1.1.tgz", + "integrity": "sha512-GpptLrs57adMSuHi3VNj0mAF8dwh36LMaYF6XyJ6JMWlVsc+t42tm1HSEDmOs3A8fC9yyeisgLhsTVQokOZ0zw==", + "license": "BSD-3-Clause", + "dependencies": { + "@protobufjs/aspromise": "^1.1.1" + } + }, + "node_modules/@protobufjs/float": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/@protobufjs/float/-/float-1.0.2.tgz", + "integrity": "sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/path": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/@protobufjs/path/-/path-1.1.2.tgz", + "integrity": "sha512-6JOcJ5Tm08dOHAbdR3GrvP+yUUfkjG5ePsHYczMFLq3ZmMkAD98cDgcT2iA1lJ9NVwFd4tH/iSSoe44YWkltEA==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/pool": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/@protobufjs/pool/-/pool-1.1.0.tgz", + "integrity": "sha512-0kELaGSIDBKvcgS4zkjz1PeddatrjYcmMWOlAuAPwAeccUrPHdUqo/J6LiymHHEiJT5NrF1UVwxY14f+fy4WQw==", + "license": "BSD-3-Clause" + }, + "node_modules/@protobufjs/utf8": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.1.tgz", + "integrity": "sha512-oOAWABowe8EAbMyWKM0tYDKi8Yaox52D+HWZhAIJqQXbqe0xI/GV7FhLWqlEKreMkfDjshR5FKgi3mnle0h6Eg==", + "license": "BSD-3-Clause" + }, + "node_modules/@rails/actioncable": { + "version": "8.1.300", + "resolved": "https://registry.npmjs.org/@rails/actioncable/-/actioncable-8.1.300.tgz", + "integrity": "sha512-zOENQsq3NM2jyBY6Z2qtZa3V/R/6OEqA+LGKixQbBMl7kk/J3FXDRcszPe74LsHNgB01jCl/DXu/xA8sHt4I/g==", + "license": "MIT" + }, "node_modules/@socket.io/component-emitter": { "version": "3.1.2", "resolved": "https://registry.npmjs.org/@socket.io/component-emitter/-/component-emitter-3.1.2.tgz", @@ -807,6 +872,16 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/centrifuge": { + "version": "5.7.0", + "resolved": "https://registry.npmjs.org/centrifuge/-/centrifuge-5.7.0.tgz", + "integrity": "sha512-Ptx7ELyVc7/KgzpadVlISTtdTWsuzumze5/vo9sH4RsvtFulJJMhmKr/cNDg6se1eKKbS6ZywIBl4eSZxqY3fw==", + "license": "MIT", + "dependencies": { + "events": "^3.3.0", + "protobufjs": "^7.6.0" + } + }, "node_modules/cluster-key-slot": { "version": "1.1.2", "resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz", @@ -1111,6 +1186,15 @@ "node": ">= 0.6" } }, + "node_modules/events": { + "version": "3.3.0", + "resolved": "https://registry.npmjs.org/events/-/events-3.3.0.tgz", + "integrity": "sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==", + "license": "MIT", + "engines": { + "node": ">=0.8.x" + } + }, "node_modules/express": { "version": "4.22.2", "resolved": "https://registry.npmjs.org/express/-/express-4.22.2.tgz", @@ -1396,6 +1480,12 @@ "integrity": "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==", "license": "MIT" }, + "node_modules/long": { + "version": "5.3.2", + "resolved": "https://registry.npmjs.org/long/-/long-5.3.2.tgz", + "integrity": "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA==", + "license": "Apache-2.0" + }, "node_modules/math-intrinsics": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz", @@ -1568,6 +1658,29 @@ "integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==", "license": "MIT" }, + "node_modules/protobufjs": { + "version": "7.6.4", + "resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.4.tgz", + "integrity": "sha512-RJJPTTpvFfHcWLkIa2JFWK4XvtSzS0yEWDmunqHXli1h3JlkbcQZXDZdcWxv+JK3Xsl5/UFDPZ0iGm7DAengYw==", + "hasInstallScript": true, + "license": "BSD-3-Clause", + "dependencies": { + "@protobufjs/aspromise": "^1.1.2", + "@protobufjs/base64": "^1.1.2", + "@protobufjs/codegen": "^2.0.5", + "@protobufjs/eventemitter": "^1.1.1", + "@protobufjs/fetch": "^1.1.1", + "@protobufjs/float": "^1.0.2", + "@protobufjs/path": "^1.1.2", + "@protobufjs/pool": "^1.1.0", + "@protobufjs/utf8": "^1.1.1", + "@types/node": ">=13.7.0", + "long": "^5.3.2" + }, + "engines": { + "node": ">=12.0.0" + } + }, "node_modules/proxy-addr": { "version": "2.0.7", "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz", diff --git a/backend/package.json b/backend/package.json index fb52b33..950f704 100644 --- a/backend/package.json +++ b/backend/package.json @@ -23,6 +23,8 @@ "bench:idle:multi": "tsx src/bench/idle-multi.ts", "bench:jitter:multi": "tsx src/bench/jitter-multi.ts", "bench:whispers:multi": "tsx src/bench/whispers-multi.ts", + "bench:preflight": "tsx src/bench/preflight.ts", + "bench:fleet": "tsx src/bench/preflight.ts status", "bench:rebaseline": "tsx src/bench/rebaseline.ts", "bench:rebaseline:history": "tsx src/bench/rebaseline-history.ts", "bench:throughput:multi": "tsx src/bench/throughput-multi.ts", @@ -41,8 +43,10 @@ }, "dependencies": { "@anycable/core": "^1.1.6", + "@rails/actioncable": "^8.1.300", "@socket.io/redis-adapter": "^8.3.0", "@socket.io/redis-streams-adapter": "^0.3.1", + "centrifuge": "^5.7.0", "express": "^4.21.0", "ioredis": "^5.10.1", "nats": "^2.29.3", diff --git a/backend/results/rails-2026-06-27.json b/backend/results/rails-2026-06-27.json new file mode 100644 index 0000000..1c5e052 --- /dev/null +++ b/backend/results/rails-2026-06-27.json @@ -0,0 +1,242 @@ +{ + "benchmark": "rails-actioncable", + "page": "https://anycable.io/compare/rails-actioncable", + "capturedAt": "2026-06-27", + "hardware": "Railway shared-tenant window; Rails 8.1, Puma 8 workers x 5 threads; anycable-go gateway + Rails gRPC RPC (anycable-rails 1.6)", + "targets": [ + "Action Cable (Redis adapter)", + "Solid Cable (DB adapter)", + "AnyCable (RPC mode, anycable-go gateway)" + ], + "results": { + "latency": { + "solidcable-1k": { + "clients": 1000, + "connectedClients": 1000, + "deliveryRatePct": 100, + "connectFailures": 0, + "latencyRawMs": { + "avg": 62, + "p50": 62, + "p95": 108, + "p99": 115, + "max": 148 + } + }, + "solidcable-5k": { + "clients": 5000, + "connectedClients": 5000, + "deliveryRatePct": 100, + "connectFailures": 0, + "latencyRawMs": { + "avg": 85, + "p50": 86, + "p95": 141, + "p99": 163, + "max": 276 + } + }, + "actioncable-1k": { + "clients": 1000, + "connectedClients": 1000, + "deliveryRatePct": 100, + "connectFailures": 0, + "latencyRawMs": { + "avg": 19, + "p50": 18, + "p95": 30, + "p99": 70, + "max": 96 + } + }, + "actioncable-5k": { + "clients": 5000, + "connectedClients": 5000, + "deliveryRatePct": 100, + "connectFailures": 0, + "latencyRawMs": { + "avg": 61, + "p50": 58, + "p95": 109, + "p99": 245, + "max": 309 + } + }, + "anycable-1k": { + "clients": 1000, + "connectedClients": 1000, + "deliveryRatePct": 100, + "connectFailures": 0, + "latencyRawMs": { + "avg": 15, + "p50": 15, + "p95": 25, + "p99": 32, + "max": 50 + } + }, + "anycable-5k": { + "clients": 5000, + "connectedClients": 5000, + "deliveryRatePct": 100, + "connectFailures": 0, + "latencyRawMs": { + "avg": 175, + "p50": 225, + "p95": 446, + "p99": 685, + "max": 712 + } + } + }, + "jitter": { + "solidcable-5k": { + "clients": 5000, + "connectedClients": 5000, + "deliveryRatePct": 28.05, + "connectFailures": 0, + "latencyRawMs": { + "avg": 84, + "p50": 84, + "p95": 151, + "p99": 183, + "max": 216 + } + }, + "actioncable-5k": { + "clients": 5000, + "connectedClients": 5000, + "deliveryRatePct": 26.7, + "connectFailures": 0, + "latencyRawMs": { + "avg": 51, + "p50": 45, + "p95": 101, + "p99": 239, + "max": 270 + } + }, + "anycable-5k": { + "clients": 5000, + "connectedClients": 5000, + "deliveryRatePct": 100, + "connectFailures": 0, + "latencyRawMs": { + "avg": 684, + "p50": 230, + "p95": 3908, + "p99": 5976, + "max": 8862 + } + } + }, + "capacityUnderLoad": { + "solidcable-10k": { + "clients": 10000, + "connectedClients": 6003, + "deliveryRatePct": 0, + "connectFailures": 0, + "latencyRawMs": { + "avg": 0, + "p50": 0, + "p95": 0, + "p99": 0, + "max": 0 + } + }, + "actioncable-10k": { + "clients": 10000, + "connectedClients": 6001, + "deliveryRatePct": 0, + "connectFailures": 0, + "latencyRawMs": { + "avg": 0, + "p50": 0, + "p95": 0, + "p99": 0, + "max": 0 + } + }, + "anycable-10k": { + "clients": 10000, + "connectedClients": 10000, + "deliveryRatePct": 100, + "connectFailures": 0, + "latencyRawMs": { + "avg": 304, + "p50": 255, + "p95": 705, + "p99": 921, + "max": 9764 + } + } + }, + "idle": { + "solidcable": { + "connected": 52000, + "welcomed": 52000, + "subscribed": 52000, + "failed": 0 + }, + "actioncable": { + "connected": 52000, + "welcomed": 52000, + "subscribed": 52000, + "failed": 0 + }, + "anycable": { + "connected": 156000, + "welcomed": 156000, + "subscribed": 156000, + "failed": 0 + } + }, + "avalanche": { + "solidcable-5k": { + "clients": 5000, + "initiallyConnected": 5000, + "disconnected": 5000, + "reconnected": 4785, + "reconnectRatePct": 95.7, + "neverReconnected": 215, + "recoveryTimeMs": 7640, + "reconnectMs": { + "p50": 4460, + "p95": 7137, + "p99": 7604, + "max": 7744 + } + }, + "actioncable-5k": { + "clients": 5000, + "initiallyConnected": 5000, + "disconnected": 5000, + "reconnected": 4813, + "reconnectRatePct": 96.26, + "neverReconnected": 187, + "recoveryTimeMs": 7526, + "reconnectMs": { + "p50": 4379, + "p95": 7091, + "p99": 7567, + "max": 7750 + } + }, + "anycable-5k": { + "clients": 5000, + "initiallyConnected": 5000, + "disconnected": 0, + "reconnected": 0, + "reconnectRatePct": 0, + "neverReconnected": 5000, + "recoveryTimeMs": 0, + "reconnectMs": { + "p50": 0, + "p95": 0, + "p99": 0, + "max": 0 + } + } + } + } +} \ No newline at end of file diff --git a/backend/results/rails-capacity-break-2026-06-28.json b/backend/results/rails-capacity-break-2026-06-28.json new file mode 100644 index 0000000..a63d86a --- /dev/null +++ b/backend/results/rails-capacity-break-2026-06-28.json @@ -0,0 +1,225 @@ +{ + "test": "idle-capacity-to-break", + "date": "2026-06-28", + "box": "Railway, 32GB plan limit per service, one shared-tenant window", + "load": "13 sharded bench-runners (Puma/Falcon ceilings); AnyCable rerun with 50 runners to 600K", + "in_process_config": "Puma WEB_CONCURRENCY=8 x RAILS_MAX_THREADS=5; Falcon COUNT=8", + "notes": [ + "Connections ramped per-shard until the holding box failed (connected<95% target).", + "Puma adapters (Action Cable, Solid Cable) hard-cap ~52K at only ~2.5GB RAM: an 8-worker file-descriptor ceiling, not memory.", + "AsyncCable on Falcon is memory-bound: ~97K held at ~27GB (fibers ~290KB/conn, ~6x the others).", + "AnyCable (anycable-go gateway) held 600K with 0 failures on a 50-runner fleet; not broken. ~47KB/conn, memory wall near ~700K on the 32GB box. ~11x the Puma adapters.", + "All four hold 10K subscribers at 100% delivery (see rails-sharded-2026-06-28.json).", + "Per-connection RAM: Action Cable ~45KB, Solid Cable ~50KB, AnyCable ~47KB, AsyncCable/Falcon ~290KB. Puma's low ceiling is the per-worker fd wall reached well below the memory limit." + ], + "results": { + "solidcable": { + "rows": [ + { + "per_shard": 4000, + "target": 52000, + "connected": 52000, + "failed": 0, + "shards_ok": 13, + "shards_err": 0, + "peak_mb": 2516.3, + "elapsed_s": 113.5, + "window": [ + "2026-06-28T16:32:12.314Z", + "2026-06-28T16:33:44.932Z" + ] + }, + { + "per_shard": 8000, + "target": 104000, + "connected": 51993, + "failed": 52007, + "shards_ok": 13, + "shards_err": 0, + "peak_mb": 2977.9, + "elapsed_s": 130.3, + "window": [ + "2026-06-28T16:34:20.845Z", + "2026-06-28T16:36:10.305Z" + ] + } + ], + "done": true, + "broke": true + }, + "actioncable": { + "rows": [ + { + "per_shard": 4000, + "target": 52000, + "connected": 52000, + "failed": 0, + "shards_ok": 13, + "shards_err": 0, + "peak_mb": 2310.3, + "elapsed_s": 113.2, + "window": [ + "2026-06-28T16:36:51.103Z", + "2026-06-28T16:38:23.447Z" + ] + }, + { + "per_shard": 8000, + "target": 104000, + "connected": 50722, + "failed": 53278, + "shards_ok": 13, + "shards_err": 0, + "peak_mb": 2682.3, + "elapsed_s": 130.4, + "window": [ + "2026-06-28T16:38:59.231Z", + "2026-06-28T16:40:48.698Z" + ] + } + ], + "done": true, + "broke": true + }, + "asynccable": { + "rows": [ + { + "per_shard": 4000, + "target": 52000, + "connected": 51713, + "failed": 287, + "shards_ok": 13, + "shards_err": 0, + "peak_mb": 13413.7, + "elapsed_s": 113.5, + "window": [ + "2026-06-28T16:41:29.618Z", + "2026-06-28T16:43:02.215Z" + ] + }, + { + "per_shard": 8000, + "target": 104000, + "connected": 97390, + "failed": 6610, + "shards_ok": 13, + "shards_err": 0, + "peak_mb": 27797.1, + "elapsed_s": 130.5, + "window": [ + "2026-06-28T16:43:38.123Z", + "2026-06-28T16:45:27.668Z" + ] + } + ], + "done": true, + "broke": true + }, + "anycable": { + "rows": [ + { + "per_shard": 4000, + "target": 52000, + "connected": 52000, + "failed": 0, + "shards_ok": 13, + "shards_err": 0, + "peak_mb": 1874.4, + "elapsed_s": 115.2, + "window": [ + "2026-06-28T16:46:08.666Z", + "2026-06-28T16:47:41.190Z" + ] + }, + { + "per_shard": 8000, + "target": 104000, + "connected": 104000, + "failed": 0, + "shards_ok": 13, + "shards_err": 0, + "peak_mb": 4594.4, + "elapsed_s": 130.4, + "window": [ + "2026-06-28T16:48:18.857Z", + "2026-06-28T16:50:08.328Z" + ] + }, + { + "per_shard": 12000, + "target": 156000, + "connected": 131801, + "failed": 24199, + "shards_ok": 13, + "shards_err": 0, + "peak_mb": 7453.5, + "elapsed_s": 147.5, + "window": [ + "2026-06-28T16:50:44.289Z", + "2026-06-28T16:52:50.830Z" + ] + } + ], + "done": true, + "broke": true + } + }, + "anycable_bigfleet": { + "runners": 50, + "rows": [ + { + "per_shard": 4000, + "target": 200000, + "connected": 200000, + "failed": 0, + "shards_ok": 50, + "shards_err": 0, + "per_shard_min": 4000, + "per_shard_max": 4000, + "gateway_peak_mb": 7358.3, + "rpc_peak_mb": 138.2, + "elapsed_s": 138.3, + "window": [ + "2026-06-28T17:32:53.792Z", + "2026-06-28T17:34:50.836Z" + ] + }, + { + "per_shard": 8000, + "target": 400000, + "connected": 400000, + "failed": 0, + "shards_ok": 50, + "shards_err": 0, + "per_shard_min": 8000, + "per_shard_max": 8000, + "gateway_peak_mb": 17057.2, + "rpc_peak_mb": 141.3, + "elapsed_s": 189.1, + "window": [ + "2026-06-28T17:35:27.157Z", + "2026-06-28T17:38:15.128Z" + ] + }, + { + "per_shard": 12000, + "target": 600000, + "connected": 600000, + "failed": 0, + "shards_ok": 50, + "shards_err": 0, + "per_shard_min": 12000, + "per_shard_max": 12000, + "gateway_peak_mb": 27657.8, + "rpc_peak_mb": 143.5, + "elapsed_s": 240.1, + "window": [ + "2026-06-28T17:38:51.275Z", + "2026-06-28T17:42:30.271Z" + ] + } + ], + "broke_at": null, + "note": "AnyCable held 600K idle with 0 failures across all 50 shards; not broken (fleet-limited at 50x12K). Gateway RAM scaled linearly to 27GB at 600K (27657.8 MB = 27.0 GB; ~47KB/conn gross), ~84% of the 32GB box, so the real wall is ~700K, memory-bound. The RPC backend stayed flat at ~140MB. The earlier 132K (13 runners) was a load-fleet limit, not anycable-go." + } +} \ No newline at end of file diff --git a/backend/results/rails-sharded-2026-06-28.json b/backend/results/rails-sharded-2026-06-28.json new file mode 100644 index 0000000..63ceed8 --- /dev/null +++ b/backend/results/rails-sharded-2026-06-28.json @@ -0,0 +1,195 @@ +{ + "benchmark": "rails-actioncable (sharded re-run)", + "capturedAt": "2026-06-28", + "method": "13 bench-runner shards (~77-770 cables/shard) via jitter-multi; corrects single-runner event-loop saturation", + "targets": [ + "Action Cable (Redis/Puma)", + "Solid Cable (DB/Puma)", + "AsyncCable (Falcon)", + "AnyCable (anycable-go RPC)" + ], + "latency": { + "solidcable": { + "1k": { + "p50": 62, + "p95": 110, + "p99": 119 + }, + "5k": { + "p50": 74, + "p95": 134, + "p99": 164 + } + }, + "actioncable": { + "1k": { + "p50": 9, + "p95": 17, + "p99": 47 + }, + "5k": { + "p50": 13, + "p95": 27, + "p99": 57 + } + }, + "asynccable": { + "1k": { + "p50": 11, + "p95": 17, + "p99": 80 + }, + "5k": { + "p50": 20, + "p95": 35, + "p99": 71 + } + }, + "anycable": { + "1k": { + "p50": 4, + "p95": 7, + "p99": 23 + }, + "5k": { + "p50": 7, + "p95": 11, + "p99": 31 + } + } + }, + "jitter": { + "solidcable": { + "deliveryPct": 78.08, + "lat": { + "p50": 71, + "p95": 125, + "p99": 153 + } + }, + "actioncable": { + "deliveryPct": 78.14, + "lat": { + "p50": 12, + "p95": 26, + "p99": 53 + } + }, + "asynccable": { + "deliveryPct": 78.14, + "lat": { + "p50": 18, + "p95": 34, + "p99": 82 + } + }, + "anycable": { + "deliveryPct": 99.92, + "lat": { + "p50": 7, + "p95": 4030, + "p99": 5964 + } + } + }, + "capacity10k": { + "solidcable": { + "connected": 10010, + "deliveryPct": 100, + "received": 1000994, + "expected": 1001000, + "trulyLost": 6, + "lat": { + "p50": 88, + "p95": 156, + "p99": 200 + } + }, + "actioncable": { + "connected": 10010, + "deliveryPct": 100, + "received": 1001000, + "expected": 1001000, + "trulyLost": 0, + "lat": { + "p50": 17, + "p95": 38, + "p99": 84 + } + }, + "asynccable": { + "connected": 10010, + "deliveryPct": 100, + "received": 1000990, + "expected": 1001000, + "trulyLost": 10, + "lat": { + "p50": 32, + "p95": 62, + "p99": 112 + } + }, + "anycable": { + "connected": 10010, + "deliveryPct": 100, + "received": 1001000, + "expected": 1001000, + "trulyLost": 0, + "lat": { + "p50": 11, + "p95": 19, + "p99": 31 + } + } + }, + "idle": { + "solidcable": { + "held": 52000, + "failed": 0, + "note": "prior same-method session" + }, + "actioncable": { + "held": 52000, + "failed": 0, + "note": "prior same-method session" + }, + "anycable": { + "held": 156000, + "failed": 0, + "note": "prior same-method session" + }, + "asynccable": { + "held": 52000, + "failed": 0 + } + }, + "avalanche": { + "asynccable": { + "disconnected": 4988, + "reconnectRatePct": 96.42, + "neverReconnected": 179, + "recoveryTimeMs": 7984 + }, + "_priorSession": { + "solidcable": { + "disconnected": 5000, + "reconnectRatePct": 95.7, + "recoveryTimeMs": 7640 + }, + "actioncable": { + "disconnected": 5000, + "reconnectRatePct": 96.26, + "recoveryTimeMs": 7526 + }, + "anycable": { + "disconnected": 0, + "reconnectRatePct": null, + "recoveryTimeMs": 0 + } + } + }, + "notes": { + "anycable_remeasured": "AnyCable latency+10K re-run in isolation 2026-06-28; the batch values (5K p99=2087ms, 10K 88%) were transient shared-tenant load noise (AnyCable ran last in a 30-min back-to-back batch). Isolated: 5K p99=31ms, 10K 100%/p99=31ms.", + "anycable_jitter_tail": "AnyCable jitter p99~6s is the resume backfill (missed messages replayed late but delivered), expected and matches the nodejs page." + } +} \ No newline at end of file diff --git a/backend/src/bench-runner/known-params.ts b/backend/src/bench-runner/known-params.ts new file mode 100644 index 0000000..05eb886 --- /dev/null +++ b/backend/src/bench-runner/known-params.ts @@ -0,0 +1,182 @@ +// Central registry of query params each bench endpoint understands. +// +// Why: the runner's parsers silently fall back to defaults for any key they +// don't read. A driver sending `interval` to an endpoint that reads +// `intervalMs` runs the whole test at the default rate with no error +// (this exact bug shipped once). The registry lets the server compute +// `unknownParams` for every request and echo them back, so the driver can +// fail fast instead of trusting a run that ignored its parameters. +// +// Keep in sync with the parsers: params.ts (jitter), throughputParamsFromQuery +// and whispersParamsFromQuery in server.ts, and the ad-hoc parsing in each +// handler. Adding a query param to a handler without registering it here +// makes every driver that sends it fail loudly, which is the point. + +const INFRA = ["async"]; + +const JITTER = [ + "n", + "duration", + "jitter", + "jitterMs", + "msgs", + "interval", + "ramp", + "stream", + "samplesCap", +]; + +const THROUGHPUT = [ + "n", + "total", + "intervalMs", + "ramp", + "stream", + "drain", + "publisher", + "publisherConcurrency", + "samplesCap", +]; + +const WHISPERS = [ + "n", + "rooms", + "ramp", + "interval", + "duration", + "payload", + "roomPrefix", + "samplesCap", +]; + +const IDLE = ["n", "hold", "ramp", "stream", "shard"]; + +const AVALANCHE = ["n", "ramp", "prearm", "recoveryWait", "stream"]; + +const ANYCABLE_TARGET = [ + "cableUrl", + "broadcastUrl", + "channel", + "acProtocol", +]; + +const KNOWN: Record = { + "/bench-jitter-anycable": [ + ...JITTER, + ...ANYCABLE_TARGET, + "reconnectMode", + "reconnectBaseMs", + "clientLib", + ], + "/bench-jitter-anycable-traced": [ + ...JITTER, + "cableUrl", + "broadcastUrl", + "traceSample", + ], + "/bench-jitter-socketio": [...JITTER, "serverUrl"], + "/bench-jitter-socketio-csr": [...JITTER, "serverUrl"], + "/bench-jitter-uws": [...JITTER, "wsUrl", "httpUrl"], + "/bench-trace-anycable": [ + "cableUrl", + "broadcastUrl", + "n", + "broadcasts", + "intervalMs", + "rampPerSec", + "stream", + "includeSpans", + ], + "/bench-idle-anycable": [...IDLE, "cableUrl", "channel", "acProtocol"], + "/bench-idle-socketio": [...IDLE, "serverUrl"], + "/bench-idle-uws": [...IDLE, "wsUrl"], + "/bench-avalanche-socketio": [...AVALANCHE, "serverUrl"], + "/bench-avalanche-anycable": [ + ...AVALANCHE, + "cableUrl", + "channel", + "acProtocol", + "clientLib", + ], + "/bench-avalanche-uws": [...AVALANCHE, "wsUrl"], + "/bench-deploy-impact-socketio": [ + "n", + "ramp", + "stream", + "pubRate", + "preDeploy", + "postDeploy", + "nodes", + ], + "/bench-deploy-impact-standalone-socketio": [ + "n", + "ramp", + "stream", + "duration", + "nodes", + ], + "/bench-deploy-impact-standalone-anycable": [ + "n", + "ramp", + "stream", + "duration", + "cableUrl", + ], + "/bench-whispers-anycable": [...WHISPERS, "cableUrl"], + "/bench-whispers-socketio": [...WHISPERS, "serverUrl"], + "/bench-whispers-uws": [...WHISPERS, "wsUrl"], + "/bench-throughput-anycable": [ + ...THROUGHPUT, + ...ANYCABLE_TARGET, + "natsUrl", + "natsSubject", + ], + "/bench-throughput-socketio": [...THROUGHPUT, "serverUrl"], + "/bench-throughput-socketio-csr": [...THROUGHPUT, "serverUrl"], + "/bench-throughput-anycable-cluster": [ + ...THROUGHPUT, + "cableUrlA", + "cableUrlB", + "broadcastUrl", + "natsUrl", + "natsSubject", + ], + "/bench-throughput-socketio-redis": [ + ...THROUGHPUT, + "subscriberUrlA", + "subscriberUrlB", + ], + "/bench-throughput-uws": [...THROUGHPUT, "wsUrl", "httpUrl"], + "/bench-benchi-anycable": [ + "c", + "r", + "d", + "S", + "s", + "quiet", + "drainTimeout", + "maxInflight", + "publishWorkers", + "publishBatch", + "tolerance", + "seed", + ], +}; + +const KNOWN_SETS: Record> = Object.fromEntries( + Object.entries(KNOWN).map(([path, keys]) => [ + path, + new Set([...keys, ...INFRA]), + ]), +); + +// Query keys the given endpoint would silently ignore. Empty array for +// endpoints not in the registry (nothing to check against). +export function unknownParamsFor( + path: string, + query: Record, +): string[] { + const known = KNOWN_SETS[path]; + if (!known) return []; + return Object.keys(query).filter((k) => !known.has(k)); +} diff --git a/backend/src/bench-runner/server.ts b/backend/src/bench-runner/server.ts index 993c4fc..68cae2f 100644 --- a/backend/src/bench-runner/server.ts +++ b/backend/src/bench-runner/server.ts @@ -18,6 +18,7 @@ import express from "express"; import { spawn } from "node:child_process"; import { getJob, startJob } from "../lib/core/job-queue.js"; +import { unknownParamsFor } from "./known-params.js"; import { paramsFromQuery } from "../lib/core/params.js"; import { @@ -29,6 +30,7 @@ import { runJitterAnycableTraced } from "../lib/jitter-anycable-traced.js"; import { runAnycableTrace } from "../lib/anycable-trace.js"; import { runIdleAnycable, runIdleSocketio, runIdleUws } from "../lib/idle-runner.js"; import { runAvalancheSocketio } from "../lib/avalanche-runner.js"; +import { runAvalancheAnycable } from "../lib/avalanche-anycable-runner.js"; import { runDeployImpactSocketio } from "../lib/deploy-impact-runner.js"; import { runStandaloneDeployImpactSocketio } from "../lib/standalone-deploy-impact-runner.js"; import { runStandaloneDeployImpactAnycable } from "../lib/standalone-deploy-impact-anycable-runner.js"; @@ -121,14 +123,30 @@ app.get("/health", (_req, res) => // Wrap each handler in this so we get both modes for free. `?async=1` // switches the response to 202 {jobId} + background execution; without // it the endpoint behaves identically to before this change. +// +// The 202 body also echoes `effectiveParams` (what the handler actually +// parsed) and `unknownParams` (query keys this endpoint would silently +// ignore, per known-params.ts). Drivers verify both before letting a +// paid run proceed: a key the runner ignores, or a value that fell back +// to a default, means the test is about to measure the wrong thing. async function respondAsync( req: express.Request, res: express.Response, run: () => Promise, + effectiveParams?: Record, ): Promise { + const unknownParams = unknownParamsFor( + req.path, + req.query as Record, + ); + if (unknownParams.length > 0) { + console.warn( + `[params] ${req.path} ignoring unknown query params: ${unknownParams.join(", ")}`, + ); + } if (req.query.async === "1") { const jobId = startJob(run); - res.status(202).json({ jobId }); + res.status(202).json({ jobId, effectiveParams, unknownParams }); return; } try { @@ -170,12 +188,39 @@ app.post("/bench-jitter-anycable", async (req, res) => { const params = paramsFromQuery(req); const cableUrl = (req.query.cableUrl as string) || ANYCABLE_URL; const broadcastUrl = (req.query.broadcastUrl as string) || ANYCABLE_BROADCAST_URL; - await respondAsync(req, res, () => - runJitterAnycable(params, { - cableUrl, - broadcastUrl, - broadcastSecret: ANYCABLE_BROADCAST_SECRET || undefined, - }), + // `?channel=BenchmarkChannel&acProtocol=actioncable-v1-json` targets a real + // Rails app (Action Cable / Solid Cable); the defaults target anycable-go's + // $pubsub channel over the extended protocol. + const channel = (req.query.channel as string) || undefined; + const acProtocol = (req.query.acProtocol as string) || undefined; + // `?reconnectMode=tuned` applies the uniform aggressive reconnect profile to + // whichever client is in use (default = each client's stock backoff). Run both + // and report side by side: default = real UX, tuned = server resume ceiling. + const reconnectMode = + (req.query.reconnectMode as string) === "tuned" ? "tuned" : "default"; + // `?reconnectBaseMs=200` sets the tuned profile's first-attempt base delay. + const reconnectBaseMs = req.query.reconnectBaseMs + ? parseInt(req.query.reconnectBaseMs as string, 10) + : undefined; + // `?clientLib=actioncable` drives the official @rails/actioncable client + // (for Action Cable / Solid Cable / Async::Cable); default @anycable/core. + const clientLib = + (req.query.clientLib as string) === "actioncable" ? "actioncable" : undefined; + await respondAsync( + req, + res, + () => + runJitterAnycable(params, { + cableUrl, + broadcastUrl, + broadcastSecret: ANYCABLE_BROADCAST_SECRET || undefined, + channel, + acProtocol, + reconnectMode, + reconnectBaseMs, + clientLib, + }), + { ...params, cableUrl, channel, acProtocol, reconnectMode, reconnectBaseMs, clientLib }, ); }); @@ -243,8 +288,11 @@ app.post("/bench-jitter-anycable-traced", async (req, res) => { app.post("/bench-jitter-socketio", async (req, res) => { const params = paramsFromQuery(req); const serverUrl = (req.query.serverUrl as string) || SOCKETIO_URL; - await respondAsync(req, res, () => - runJitterSocketio(params, { serverUrl }), + await respondAsync( + req, + res, + () => runJitterSocketio(params, { serverUrl }), + { ...params, serverUrl }, ); }); @@ -254,8 +302,11 @@ app.post("/bench-jitter-socketio", async (req, res) => { app.post("/bench-jitter-socketio-csr", async (req, res) => { const params = paramsFromQuery(req); const serverUrl = (req.query.serverUrl as string) || SOCKETIO_URL; - await respondAsync(req, res, () => - runJitterSocketioCsr(params, { serverUrl }), + await respondAsync( + req, + res, + () => runJitterSocketioCsr(params, { serverUrl }), + { ...params, serverUrl }, ); }); @@ -266,8 +317,11 @@ app.post("/bench-jitter-uws", async (req, res) => { const params = paramsFromQuery(req); const wsUrl = (req.query.wsUrl as string) || UWS_WS_URL; const httpUrl = (req.query.httpUrl as string) || UWS_HTTP_URL; - await respondAsync(req, res, () => - runJitterUws(params, { serverWsUrl: wsUrl, serverHttpUrl: httpUrl }), + await respondAsync( + req, + res, + () => runJitterUws(params, { serverWsUrl: wsUrl, serverHttpUrl: httpUrl }), + { ...params, wsUrl, httpUrl }, ); }); @@ -285,9 +339,19 @@ app.post("/bench-idle-anycable", async (req, res) => { const stream = (req.query.stream as string) || "idle-probe"; const shardLabel = (req.query.shard as string) || undefined; const cableUrl = (req.query.cableUrl as string) || ANYCABLE_URL; - - await respondAsync(req, res, () => - runIdleAnycable({ n, holdSec, rampPerSec, stream }, cableUrl, shardLabel), + const channel = (req.query.channel as string) || undefined; + const acProtocol = (req.query.acProtocol as string) || undefined; + + await respondAsync( + req, + res, + () => + runIdleAnycable( + { n, holdSec, rampPerSec, stream, channel, acProtocol }, + cableUrl, + shardLabel, + ), + { n, holdSec, rampPerSec, stream, cableUrl, channel, acProtocol }, ); }); @@ -306,8 +370,11 @@ app.post("/bench-idle-socketio", async (req, res) => { const shardLabel = (req.query.shard as string) || undefined; const serverUrl = (req.query.serverUrl as string) || SOCKETIO_URL; - await respondAsync(req, res, () => - runIdleSocketio({ n, holdSec, rampPerSec, stream }, serverUrl, shardLabel), + await respondAsync( + req, + res, + () => runIdleSocketio({ n, holdSec, rampPerSec, stream }, serverUrl, shardLabel), + { n, holdSec, rampPerSec, stream, serverUrl }, ); }); @@ -321,8 +388,11 @@ app.post("/bench-idle-uws", async (req, res) => { const shardLabel = (req.query.shard as string) || undefined; const wsUrl = (req.query.wsUrl as string) || UWS_WS_URL; - await respondAsync(req, res, () => - runIdleUws({ n, holdSec, rampPerSec, stream }, wsUrl, shardLabel), + await respondAsync( + req, + res, + () => runIdleUws({ n, holdSec, rampPerSec, stream }, wsUrl, shardLabel), + { n, holdSec, rampPerSec, stream, wsUrl }, ); }); @@ -343,11 +413,44 @@ app.post("/bench-avalanche-socketio", async (req, res) => { const stream = (req.query.stream as string) || "avalanche"; const serverUrl = (req.query.serverUrl as string) || SOCKETIO_URL; - await respondAsync(req, res, () => - runAvalancheSocketio( - { n, rampPerSec, prearmSec, recoveryWaitSec, stream }, - serverUrl, - ), + await respondAsync( + req, + res, + () => + runAvalancheSocketio( + { n, rampPerSec, prearmSec, recoveryWaitSec, stream }, + serverUrl, + ), + { n, rampPerSec, prearmSec, recoveryWaitSec, stream, serverUrl }, + ); +}); + +// Action Cable avalanche — connect N cables (AnyCable / Action Cable / Solid +// Cable), wait for an externally-triggered redeploy, measure recovery. For the +// in-process adapters (redeploy Puma) connections drop and reconnect; for +// AnyCable (redeploy the Rails RPC backend) the gateway holds them and +// `disconnected` stays ~0. `?channel=` + `?acProtocol=` select the target. +app.post("/bench-avalanche-anycable", async (req, res) => { + const n = parseInt((req.query.n as string) || "1000", 10); + const rampPerSec = parseInt((req.query.ramp as string) || "200", 10); + const prearmSec = parseInt((req.query.prearm as string) || "120", 10); + const recoveryWaitSec = parseInt((req.query.recoveryWait as string) || "240", 10); + const stream = (req.query.stream as string) || "avalanche-ac"; + const cableUrl = (req.query.cableUrl as string) || ANYCABLE_URL; + const channel = (req.query.channel as string) || undefined; + const acProtocol = (req.query.acProtocol as string) || undefined; + const clientLib = + (req.query.clientLib as string) === "actioncable" ? "actioncable" : undefined; + + await respondAsync( + req, + res, + () => + runAvalancheAnycable( + { n, rampPerSec, prearmSec, recoveryWaitSec, stream }, + { cableUrl, channel, acProtocol, clientLib }, + ), + { n, rampPerSec, prearmSec, recoveryWaitSec, stream, cableUrl, channel, acProtocol, clientLib }, ); }); @@ -494,22 +597,31 @@ function whispersParamsFromQuery(req: express.Request) { app.post("/bench-whispers-anycable", async (req, res) => { const params = whispersParamsFromQuery(req); const cableUrl = (req.query.cableUrl as string) || ANYCABLE_URL; - await respondAsync(req, res, () => runWhispersAnycable(params, cableUrl)); + await respondAsync(req, res, () => runWhispersAnycable(params, cableUrl), { + ...params, + cableUrl, + }); }); app.post("/bench-whispers-socketio", async (req, res) => { const params = whispersParamsFromQuery(req); const serverUrl = (req.query.serverUrl as string) || SOCKETIO_URL; - await respondAsync(req, res, () => runWhispersSocketio(params, { serverUrl })); + await respondAsync( + req, + res, + () => runWhispersSocketio(params, { serverUrl }), + { ...params, serverUrl }, + ); }); app.post("/bench-whispers-uws", async (req, res) => { const params = whispersParamsFromQuery(req); const serverWsUrl = (req.query.wsUrl as string) || UWS_WS_URL; - await respondAsync(req, res, () => - runWhispersUws(params, { serverWsUrl }), - ); + await respondAsync(req, res, () => runWhispersUws(params, { serverWsUrl }), { + ...params, + wsUrl: serverWsUrl, + }); }); // uWebSockets.js avalanche — same shape and methodology as the Socket.io @@ -526,11 +638,15 @@ app.post("/bench-avalanche-uws", async (req, res) => { const stream = (req.query.stream as string) || "avalanche-uws"; const wsUrl = (req.query.wsUrl as string) || UWS_WS_URL; - await respondAsync(req, res, () => - runAvalancheUws( - { n, rampPerSec, prearmSec, recoveryWaitSec, stream }, - wsUrl, - ), + await respondAsync( + req, + res, + () => + runAvalancheUws( + { n, rampPerSec, prearmSec, recoveryWaitSec, stream }, + wsUrl, + ), + { n, rampPerSec, prearmSec, recoveryWaitSec, stream, wsUrl }, ); }); @@ -563,30 +679,47 @@ app.post("/bench-throughput-anycable", async (req, res) => { const natsUrl = (req.query.natsUrl as string) || ANYCABLE_NATS_URL || undefined; const natsSubject = (req.query.natsSubject as string) || ANYCABLE_NATS_SUBJECT || undefined; - await respondAsync(req, res, () => - runThroughputAnycable(params, { - cableUrl, - broadcastUrl, - broadcastSecret: ANYCABLE_BROADCAST_SECRET || undefined, - natsUrl, - natsSubject, - }), + // `?channel=BenchmarkChannel&acProtocol=actioncable-v1-json` targets a real + // Rails app over the base protocol; defaults keep the anycable-go $pubsub + // channel over the extended protocol. + const channel = (req.query.channel as string) || undefined; + const acProtocol = (req.query.acProtocol as string) || undefined; + await respondAsync( + req, + res, + () => + runThroughputAnycable(params, { + cableUrl, + broadcastUrl, + broadcastSecret: ANYCABLE_BROADCAST_SECRET || undefined, + natsUrl, + natsSubject, + channel, + acProtocol, + }), + { ...params, cableUrl, channel, acProtocol }, ); }); app.post("/bench-throughput-socketio", async (req, res) => { const params = throughputParamsFromQuery(req, `tp-sio-${Date.now()}`); const serverUrl = (req.query.serverUrl as string) || SOCKETIO_URL; - await respondAsync(req, res, () => - runThroughputSocketio(params, { serverUrl }), + await respondAsync( + req, + res, + () => runThroughputSocketio(params, { serverUrl }), + { ...params, serverUrl }, ); }); app.post("/bench-throughput-socketio-csr", async (req, res) => { const params = throughputParamsFromQuery(req, `tp-csr-${Date.now()}`); const serverUrl = (req.query.serverUrl as string) || SOCKETIO_URL; - await respondAsync(req, res, () => - runThroughputSocketioCsr(params, { serverUrl }), + await respondAsync( + req, + res, + () => runThroughputSocketioCsr(params, { serverUrl }), + { ...params, serverUrl }, ); }); @@ -624,8 +757,11 @@ app.post("/bench-throughput-socketio-redis", async (req, res) => { const params = throughputParamsFromQuery(req, `tp-redis-${Date.now()}`); const subscriberUrlA = (req.query.subscriberUrlA as string) || SOCKETIO_REDIS_URL_A; const subscriberUrlB = (req.query.subscriberUrlB as string) || SOCKETIO_REDIS_URL_B; - await respondAsync(req, res, () => - runThroughputSocketioRedis(params, { subscriberUrlA, subscriberUrlB }), + await respondAsync( + req, + res, + () => runThroughputSocketioRedis(params, { subscriberUrlA, subscriberUrlB }), + { ...params, subscriberUrlA, subscriberUrlB }, ); }); @@ -633,11 +769,15 @@ app.post("/bench-throughput-uws", async (req, res) => { const params = throughputParamsFromQuery(req, `tp-uws-${Date.now()}`); const wsUrl = (req.query.wsUrl as string) || UWS_WS_URL; const httpUrl = (req.query.httpUrl as string) || UWS_HTTP_URL; - await respondAsync(req, res, () => - runThroughputUws(params, { - serverWsUrl: wsUrl, - serverHttpUrl: httpUrl, - }), + await respondAsync( + req, + res, + () => + runThroughputUws(params, { + serverWsUrl: wsUrl, + serverHttpUrl: httpUrl, + }), + { ...params, wsUrl, httpUrl }, ); }); diff --git a/backend/src/bench/avalanche-multi-anycable.ts b/backend/src/bench/avalanche-multi-anycable.ts new file mode 100644 index 0000000..62c21f3 --- /dev/null +++ b/backend/src/bench/avalanche-multi-anycable.ts @@ -0,0 +1,251 @@ +// Multi-shard avalanche driver for AnyCable / Rails Action Cable targets. +// +// Mirrors avalanche-multi-uws.ts but drives the /bench-avalanche-anycable +// endpoint, so the reconnect storm after a real Railway redeploy is generated +// by MANY bench-runner shards in parallel instead of one (a single Node +// process reconnecting thousands of cables is itself load-generator-limited). +// Each shard ramps PER_SHARD_N clients to the same Rails target (channel + +// acProtocol + cableUrl), then the coordinator fires ONE serviceInstanceRedeploy +// against TARGET_SERVICE_ID; every shard sees the disconnect storm at the same +// wall-clock moment, counts its own reconnects, and we aggregate. +// +// prearm/recovery are kept short by default so the blocking shard request stays +// under Railway's ~5-minute public-proxy timeout (reconnect-to-95% is ~8s in +// practice, so a 120s recovery window is ample). +// +// Usage: +// SHARDS=https://bench-runner-2-production.up.railway.app,... \ +// PER_SHARD_N=250 \ +// CABLE_URL=ws://rails-actioncable.railway.internal:3000/cable \ +// CHANNEL=BenchmarkChannel AC_PROTOCOL=actioncable-v1-json \ +// TARGET_SERVICE_ID= TARGET_ENV_ID= \ +// tsx src/bench/avalanche-multi-anycable.ts + +import { readFileSync, writeFileSync } from "fs"; +import { homedir } from "os"; +import { Agent, setGlobalDispatcher } from "undici"; + +import { percentile } from "../lib/core/stats.js"; +import { resultPath } from "../lib/core/results-dir.js"; + +setGlobalDispatcher( + new Agent({ headersTimeout: 30 * 60 * 1000, bodyTimeout: 30 * 60 * 1000 }) +); + +const shardCsv = process.env.SHARDS; +if (!shardCsv) { + console.error("SHARDS env var is required (comma-separated bench-runner URLs)"); + process.exit(1); +} +const shardUrls = shardCsv.split(",").map((s) => s.trim()).filter(Boolean); +if (shardUrls.length === 0) { + console.error("SHARDS must contain at least one URL"); + process.exit(1); +} + +const perShardN = parseInt(process.env.PER_SHARD_N || "250", 10); +const rampPerSec = parseInt(process.env.RAMP_PER_SEC || "200", 10); +const prearmSec = parseInt(process.env.PREARM_SEC || "60", 10); +const recoveryWaitSec = parseInt(process.env.RECOVERY_WAIT_SEC || "120", 10); + +const cableUrl = process.env.CABLE_URL; +if (!cableUrl) { + console.error("CABLE_URL env var is required"); + process.exit(1); +} +const channel = process.env.CHANNEL; +const acProtocol = process.env.AC_PROTOCOL; +const clientLib = process.env.CLIENT_LIB; + +const targetServiceId = process.env.TARGET_SERVICE_ID; +const targetEnvId = process.env.TARGET_ENV_ID; +if (!targetServiceId || !targetEnvId) { + console.error("TARGET_SERVICE_ID and TARGET_ENV_ID env vars are required"); + process.exit(1); +} + +const bearerToken = process.env.BENCH_RUNNER_TOKEN; + +function readRailwayToken(): string { + if (process.env.RAILWAY_TOKEN) return process.env.RAILWAY_TOKEN; + const cfg = JSON.parse(readFileSync(`${homedir()}/.railway/config.json`, "utf-8")); + return cfg.user.token; +} + +interface AvalancheShardResult { + initiallyConnected: number; + disconnected: number; + reconnected: number; + reconnectRatePct: number; + neverReconnected: number; + recoveryTimeMs: number; + reconnectMs: { p50: number; p95: number; p99: number; max: number }; +} + +const totalTarget = perShardN * shardUrls.length; +const rampSec = Math.ceil(perShardN / rampPerSec); +const shardTimeoutMs = parseInt( + process.env.SHARD_TIMEOUT_MS || + String((rampSec + 5 + prearmSec + recoveryWaitSec + 90) * 1000), + 10 +); + +console.log(`Multi-shard AnyCable/Rails avalanche: ${shardUrls.length} shards × ${perShardN} = ${totalTarget} clients`); +console.log(` ramp: ${rampPerSec}/s per shard (~${rampSec}s)`); +console.log(` prearm: ${prearmSec}s recoveryWait: ${recoveryWaitSec}s shard timeout: ${(shardTimeoutMs / 1000).toFixed(0)}s`); +console.log(` target: ${cableUrl} channel=${channel} proto=${acProtocol}`); +console.log(` redeploy service: ${targetServiceId}\n`); + +interface ShardOutcome { + label: string; + ok: boolean; + reason?: string; + result?: AvalancheShardResult; +} + +async function runShard(url: string, label: string): Promise { + const qs = new URLSearchParams({ + n: String(perShardN), + ramp: String(rampPerSec), + prearm: String(prearmSec), + recoveryWait: String(recoveryWaitSec), + stream: `avalanche-ac-multi-${label}-${Date.now()}`, + cableUrl: cableUrl as string, + }); + if (channel) qs.set("channel", channel); + if (acProtocol) qs.set("acProtocol", acProtocol); + if (clientLib) qs.set("clientLib", clientLib); + + const ctrl = new AbortController(); + const t = setTimeout(() => ctrl.abort(), shardTimeoutMs); + try { + const res = await fetch(`${url}/bench-avalanche-anycable?${qs.toString()}`, { + method: "POST", + headers: bearerToken ? { Authorization: `Bearer ${bearerToken}` } : {}, + signal: ctrl.signal, + }); + if (!res.ok) { + console.log(` ✗ ${label}: ${res.status} ${res.statusText}`); + return { label, ok: false, reason: `HTTP ${res.status}` }; + } + const result = (await res.json()) as AvalancheShardResult; + console.log( + ` ✓ ${label}: connected=${result.initiallyConnected} reconnected=${result.reconnected} (${result.reconnectRatePct}%) recovery=${result.recoveryTimeMs}ms` + ); + return { label, ok: true, result }; + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + console.log(` ✗ ${label}: ${msg}`); + return { label, ok: false, reason: msg }; + } finally { + clearTimeout(t); + } +} + +async function fireRedeploy(): Promise { + const token = readRailwayToken(); + const res = await fetch("https://backboard.railway.com/graphql/v2", { + method: "POST", + headers: { Authorization: `Bearer ${token}`, "Content-Type": "application/json" }, + body: JSON.stringify({ + query: + "mutation R($id: String!, $env: String!) { serviceInstanceRedeploy(serviceId: $id, environmentId: $env) }", + variables: { id: targetServiceId, env: targetEnvId }, + }), + }); + const data = (await res.json()) as { + data?: { serviceInstanceRedeploy?: boolean }; + errors?: unknown[]; + }; + if (data.errors) { + console.error(" ! redeploy mutation failed:", JSON.stringify(data.errors)); + return false; + } + return data.data?.serviceInstanceRedeploy === true; +} + +const startedAt = new Date(); +console.log(`Test started at ${startedAt.toISOString()}\n`); + +const shardPromises = shardUrls.map((u, i) => runShard(u, `shard-${i + 1}`)); + +const triggerInSec = rampSec + 10; +console.log(`(ramping in parallel; firing redeploy in ${triggerInSec}s)`); +await new Promise((r) => setTimeout(r, triggerInSec * 1000)); + +console.log(`\n>>> Firing redeploy on ${targetServiceId}...`); +const ok = await fireRedeploy(); +if (!ok) { + console.error("Redeploy mutation failed; bailing."); + await Promise.allSettled(shardPromises); + process.exit(1); +} +console.log(`Redeploy triggered. Awaiting shard results...\n`); + +const settled = await Promise.allSettled(shardPromises); +const endedAt = new Date(); +console.log(`\nTest ended at ${endedAt.toISOString()}`); +console.log(`Total elapsed: ${((endedAt.getTime() - startedAt.getTime()) / 1000).toFixed(1)}s`); + +const ok_results: AvalancheShardResult[] = []; +const errors: string[] = []; +settled.forEach((s) => { + if (s.status === "fulfilled") { + if (s.value.ok && s.value.result) ok_results.push(s.value.result); + else errors.push(`${s.value.label}: ${s.value.reason}`); + } else { + errors.push(String(s.reason)); + } +}); + +const totals = ok_results.reduce( + (acc, r) => ({ + initiallyConnected: acc.initiallyConnected + r.initiallyConnected, + reconnected: acc.reconnected + r.reconnected, + neverReconnected: acc.neverReconnected + r.neverReconnected, + disconnected: acc.disconnected + r.disconnected, + }), + { initiallyConnected: 0, reconnected: 0, neverReconnected: 0, disconnected: 0 } +); + +const recoveryMs = ok_results.map((r) => r.recoveryTimeMs).sort((a, b) => a - b); +const p50s = ok_results.map((r) => r.reconnectMs.p50).sort((a, b) => a - b); +const p95s = ok_results.map((r) => r.reconnectMs.p95).sort((a, b) => a - b); +const p99s = ok_results.map((r) => r.reconnectMs.p99).sort((a, b) => a - b); + +console.log(`\n=== Aggregate (across ${ok_results.length}/${shardUrls.length} surviving shards) ===`); +console.log(` Target connections: ${totalTarget.toLocaleString()}`); +console.log(` Initially connected: ${totals.initiallyConnected.toLocaleString()}`); +console.log(` Disconnected events: ${totals.disconnected.toLocaleString()}`); +console.log(` Reconnected: ${totals.reconnected.toLocaleString()}`); +console.log( + ` Reconnect rate: ${ + totals.initiallyConnected > 0 + ? ((totals.reconnected / totals.initiallyConnected) * 100).toFixed(2) + : "—" + }%` +); +console.log(` Never reconnected: ${totals.neverReconnected.toLocaleString()}`); +console.log( + ` Recovery (time to 95%): median=${percentile(recoveryMs, 50)}ms p95=${percentile(recoveryMs, 95)}ms max=${percentile(recoveryMs, 100)}ms` +); +console.log( + ` Per-shard reconnect: median p50=${percentile(p50s, 50)} p95=${percentile(p95s, 50)} p99=${percentile(p99s, 50)}` +); + +if (errors.length > 0) { + console.log(`\n${errors.length} shard(s) errored or timed out:`); + for (const e of errors) console.log(` - ${e}`); +} + +const dump = { + startedAt: startedAt.toISOString(), + endedAt: endedAt.toISOString(), + config: { perShardN, rampPerSec, prearmSec, recoveryWaitSec, cableUrl, channel, acProtocol, totalTarget, shards: shardUrls.length }, + totals, + per_shard: ok_results, + errors, +}; +const path = resultPath(`avalanche-multi-anycable-${startedAt.toISOString().replace(/[:.]/g, "-")}.json`); +writeFileSync(path, JSON.stringify(dump, null, 2)); +console.log(`\nWrote ${path}`); diff --git a/backend/src/bench/avalanche-multi.ts b/backend/src/bench/avalanche-multi.ts index f5db0fa..b46be3b 100644 --- a/backend/src/bench/avalanche-multi.ts +++ b/backend/src/bench/avalanche-multi.ts @@ -45,6 +45,19 @@ const railwayService = process.env.RAILWAY_SERVICE || "socketio-server"; // regular one (e.g. socketio-server-csr.railway.internal:3000). const serverUrl = process.env.SERVER_URL; +// Protocol selects the bench-runner endpoint (/bench-avalanche-). +// Default to anycable for Rails RPC services, else socketio. The anycable +// endpoint also drives the Rails Action Cable / Solid Cable / Async::Cable +// targets when given channel + acProtocol + cableUrl (base or ext protocol). +const protocol = ( + process.env.PROTOCOL || + (railwayService.startsWith("rails-") ? "anycable" : "socketio") +).toLowerCase(); +// Rails target overrides, forwarded to /bench-avalanche-anycable. +const channel = process.env.CHANNEL; +const acProtocol = process.env.AC_PROTOCOL; +const cableUrl = process.env.CABLE_URL; + const scales = (process.env.SCALES || "1000,2500,5000,10000,20000") .split(",") .map((s) => parseInt(s.trim(), 10)) @@ -103,10 +116,15 @@ async function runOneScale(n: number): Promise { stream: `avalanche-${n}`, }); if (serverUrl) qs.set("serverUrl", serverUrl); + // Rails targets: forward the channel + wire protocol + WS URL so the + // anycable avalanche endpoint connects to the real Rails app. + if (channel) qs.set("channel", channel); + if (acProtocol) qs.set("acProtocol", acProtocol); + if (cableUrl) qs.set("cableUrl", cableUrl); const startedAt = Date.now(); const responsePromise = benchRunnerFetch( - `${benchRunnerUrl}/bench-avalanche-socketio?${qs.toString()}`, + `${benchRunnerUrl}/bench-avalanche-${protocol}?${qs.toString()}`, { method: "POST" } ); diff --git a/backend/src/bench/catchup-probe.ts b/backend/src/bench/catchup-probe.ts new file mode 100644 index 0000000..d9dbb4d --- /dev/null +++ b/backend/src/bench/catchup-probe.ts @@ -0,0 +1,144 @@ +// Catch-up probe: does an AnyCable subscriber recover messages it wasn't +// present for? Distinguishes the two paths that behave very differently: +// +// RESUME (old client): a client that already subscribed (has a session + +// stream offset), drops, and reconnects. AnyCable restores the session and +// replays the stream from the last offset. Expect: backfill of the gap. +// +// COLD (new client): a brand-new client subscribing for the first time, +// AFTER messages were already broadcast. No prior offset. Expect: this is +// the open question, does it catch up on the backlog or start at "now"? +// +// Runs a handful of clients against the public gateway (no runner fleet). +// Env: CABLE_URL, BROADCAST_URL, BROADCAST_SECRET, CHANNEL. +import WebSocket from "ws"; +import { createCable } from "@anycable/core"; + +const CABLE = process.env.CABLE_URL!; +const BCAST = process.env.BROADCAST_URL!; +const SECRET = process.env.BROADCAST_SECRET || "benchsecret"; +const CHANNEL = process.env.CHANNEL || "BenchmarkChannel"; + +const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); + +function mkCable(historyTimestamp?: number) { + return createCable(CABLE, { + websocketImplementation: WebSocket as unknown as typeof globalThis.WebSocket, + protocol: "actioncable-v1-ext-json" as never, + logLevel: "error" as never, + ...(historyTimestamp ? { historyTimestamp } as never : {}), + }); +} + +async function broadcast(stream: string, seqs: number[]) { + for (const seq of seqs) { + await fetch(BCAST, { + method: "POST", + headers: { "Content-Type": "application/json", Authorization: `Bearer ${SECRET}` }, + body: JSON.stringify({ stream, data: JSON.stringify({ seq, sentAt: Date.now() }) }), + }); + await sleep(40); + } +} + +// Extract our seq from whatever shape the message arrives in. +function seqOf(msg: unknown): number | null { + let m: unknown = msg; + if (typeof m === "string") { + try { m = JSON.parse(m); } catch { return null; } + } + if (m && typeof m === "object" && typeof (m as { seq?: unknown }).seq === "number") { + return (m as { seq: number }).seq; + } + return null; +} + +function subscribe(cable: ReturnType, stream: string, sink: number[]) { + const ch = cable.subscribeTo(CHANNEL, { stream_name: stream }); + ch.on("message", (msg: unknown) => { + const s = seqOf(msg); + if (s !== null) sink.push(s); + }); + return ch; +} + +function uniqSorted(a: number[]) { return [...new Set(a)].sort((x, y) => x - y); } + +async function coldCatchup(stamp: number, M: number) { + const stream = `catchup-cold-${stamp}-${M}`; + await broadcast(stream, Array.from({ length: M }, (_, i) => i + 1)); // broadcast FIRST + await sleep(1200); + const cable = mkCable(); + const got: number[] = []; + subscribe(cable, stream, got); // then subscribe (new client) + await sleep(4500); + cable.disconnect(); + const g = uniqSorted(got); + console.log(`[COLD M=${M}] new client subscribed AFTER ${M} broadcasts -> caught up ${g.length}/${M}` + + (g.length ? ` (first=${g[0]} last=${g[g.length - 1]})` : "")); + await sleep(400); +} + +async function resumeReconnect(stamp: number) { + const stream = `catchup-resume-${stamp}`; + const cable = mkCable(); + const got: number[] = []; + subscribe(cable, stream, got); + await sleep(1500); // let subscription confirm + await broadcast(stream, [1, 2, 3]); // received while connected + await sleep(1000); + const beforeDrop = uniqSorted(got); + // Force-drop the underlying socket (unclean), like a network blip / restart. + const transport = (cable as unknown as { transport?: { ws?: { terminate?: () => void; close?: () => void } } }).transport; + transport?.ws?.terminate?.() ?? transport?.ws?.close?.(); + await sleep(400); + await broadcast(stream, [4, 5, 6, 7, 8]); // sent DURING the outage + await sleep(200); + await cable.connect().catch(() => {}); // reconnect (session restore + resume) + await sleep(5000); + cable.disconnect(); + const after = uniqSorted(got); + const backfilled = [4, 5, 6, 7, 8].filter((s) => after.includes(s)); + console.log(`[RESUME] before drop: ${beforeDrop.join(",")} | sent during outage: 4,5,6,7,8 | after reconnect total: ${after.join(",")}`); + console.log(`[RESUME] gap backfilled on reconnect: ${backfilled.length}/5 (${backfilled.join(",")})`); +} + +// Same as coldCatchup, but the client is created with historyTimestamp set to +// before the broadcasts, so it explicitly requests stream history on subscribe. +async function coldWithHistory(stamp: number, M: number) { + const since = Math.floor(Date.now() / 1000) - 120; // 120s ago, before the broadcast + const stream = `catchup-hist-${stamp}-${M}`; + await broadcast(stream, Array.from({ length: M }, (_, i) => i + 1)); + await sleep(1200); + const cable = mkCable(since); + const got: number[] = []; + subscribe(cable, stream, got); + await sleep(5000); + cable.disconnect(); + const g = uniqSorted(got); + console.log(`[COLD+HIST M=${M}] new client WITH historyTimestamp -> caught up ${g.length}/${M}` + + (g.length ? ` (first=${g[0]} last=${g[g.length - 1]})` : "")); + await sleep(400); +} + +async function baseline(stamp: number) { + const stream = `catchup-base-${stamp}`; + const cable = mkCable(); + const got: number[] = []; + subscribe(cable, stream, got); + await sleep(1500); + await broadcast(stream, [1, 2, 3, 4, 5]); + await sleep(2500); + cable.disconnect(); + console.log(`[BASE ] subscribe-then-broadcast 5 -> received ${uniqSorted(got).length}/5 (sanity)`); + await sleep(400); +} + +async function main() { + console.log(`Catch-up probe -> ${CABLE} (channel ${CHANNEL})\n`); + const stamp = Date.now(); + await coldCatchup(stamp, 20); // control: cold client, no history request + await coldWithHistory(stamp, 20); // cold client that DOES request history + await coldWithHistory(stamp, 150); // ... and beyond history_limit=100 +} +main().then(() => process.exit(0)).catch((e) => { console.error(e); process.exit(1); }); diff --git a/backend/src/bench/idle-multi.ts b/backend/src/bench/idle-multi.ts index 9543cb9..f133ea1 100644 --- a/backend/src/bench/idle-multi.ts +++ b/backend/src/bench/idle-multi.ts @@ -5,134 +5,122 @@ // they don't share the ~64K ephemeral-port ceiling that limits a single // container. Total connections = sum of per-shard counts. // +// Runs through the async job protocol (enqueue + poll) like the other +// multi drivers, so long ramps and holds never hit Railway's 5-minute +// edge timeout, and the params echo verifies each shard parsed what we +// sent before the run starts. +// // After the shards finish, queries Railway metrics over the test window -// and prints ASCII charts of memory + CPU on the anycable-go service plus -// a CSV file (idle-multi-.csv) for offline plotting. +// and prints ASCII charts of memory + CPU on the target service plus +// a CSV file for offline plotting. // // Usage: -// SHARDS=https://bench-runner-1.up.railway.app,https://bench-runner-2.up.railway.app,... \ -// PER_SHARD_N=25000 HOLD_SEC=120 RAMP_PER_SEC=200 \ -// PROJECT_ID= SERVICE_ID= SERVICE_NAME=anycable-go \ +// SHARDS=https://bench-runner-2.up.railway.app,... \ +// PER_SHARD_N=10000 HOLD_SEC=120 RAMP_PER_SEC=200 \ +// TARGET=anycable \ # or socketio | uws +// PROJECT_ID= SERVICE_ID= SERVICE_NAME=anycable-go \ // tsx src/bench/idle-multi.ts // -// All metrics-related env vars are optional — if PROJECT_ID and SERVICE_ID -// aren't set, the script skips the chart and only reports aggregate counts. +// Metrics env vars are optional — without PROJECT_ID and SERVICE_ID the +// script skips the chart and only reports aggregate counts. +// +// Publishing rule: a capacity number is only a server ceiling when the +// failure is on the server side. Shards that all stop at the same count +// below PER_SHARD_N hit the load generator's wall — the script flags this +// and such a run must be re-run with more shards, never published. import { writeFileSync } from "fs"; -import { Agent, setGlobalDispatcher } from "undici"; import type { IdleResult } from "../lib/idle-runner.js"; import { fetchMetric, readRailwayToken } from "../lib/core/railway-api.js"; import { chart } from "../lib/core/chart.js"; +import { parseShardUrls } from "../lib/core/multi-shard.js"; +import { checkShardHealth } from "../lib/core/multi-shard.js"; import { resultPath } from "../lib/core/results-dir.js"; +import { runShards, type ShardSpec } from "../lib/core/shard-coordinator.js"; import { percentile } from "../lib/core/stats.js"; -// Each shard responds only after its full ramp + hold completes — at -// 50K-per-shard with a 120s hold, that's ~5 minutes per request. Bump -// the default 5-min fetch headers timeout so the coordinator doesn't -// give up before the shards finish. -setGlobalDispatcher( - new Agent({ headersTimeout: 30 * 60 * 1000, bodyTimeout: 30 * 60 * 1000 }) -); +const shardUrls = parseShardUrls(); -const shardCsv = process.env.SHARDS; -if (!shardCsv) { - console.error("SHARDS env var is required (comma-separated bench-runner URLs)"); - process.exit(1); -} -const shardUrls = shardCsv.split(",").map((s) => s.trim()).filter(Boolean); -if (shardUrls.length === 0) { - console.error("SHARDS must contain at least one URL"); - process.exit(1); -} - -const perShardN = parseInt(process.env.PER_SHARD_N || "25000", 10); +const perShardN = parseInt(process.env.PER_SHARD_N || "10000", 10); const holdSec = parseInt(process.env.HOLD_SEC || "120", 10); const rampPerSec = parseInt(process.env.RAMP_PER_SEC || "200", 10); const stream = process.env.STREAM || "idle-probe"; -// Optional override sent to each shard so the bench-runner targets a -// different anycable-go service (e.g. anycable-go-pro for the Pro variant). -const cableUrl = process.env.CABLE_URL; -// TARGET=socketio switches the test to /bench-idle-socketio (Node-based -// Socket.io). TARGET=uws targets /bench-idle-uws (uWebSockets.js). -// Defaults to anycable for backwards compatibility. const target = (process.env.TARGET || "anycable").toLowerCase(); if (target !== "anycable" && target !== "socketio" && target !== "uws") { console.error(`TARGET must be "anycable", "socketio", or "uws" (got "${target}")`); process.exit(1); } -// SERVER_URL overrides the Socket.io target (TARGET=socketio variant). -const socketioServerUrl = process.env.SERVER_URL; -// UWS_WS_URL overrides the uWS target (TARGET=uws variant). -const uwsWsUrl = process.env.UWS_WS_URL; +const endpoint = + target === "socketio" + ? "bench-idle-socketio" + : target === "uws" + ? "bench-idle-uws" + : "bench-idle-anycable"; + +// Target overrides, same names as every other driver. +const targetQuery: Record = {}; +if (target === "anycable") { + if (process.env.CABLE_URL) targetQuery.cableUrl = process.env.CABLE_URL; + if (process.env.CHANNEL) targetQuery.channel = process.env.CHANNEL; + if (process.env.AC_PROTOCOL) targetQuery.acProtocol = process.env.AC_PROTOCOL; +} +if (target === "socketio" && process.env.SERVER_URL) + targetQuery.serverUrl = process.env.SERVER_URL; +if (target === "uws" && process.env.UWS_WS_URL) + targetQuery.wsUrl = process.env.UWS_WS_URL; const totalTarget = perShardN * shardUrls.length; console.log( - `Idle multi-shard test: ${shardUrls.length} shards × ${perShardN} = ${totalTarget} connections` + `Idle multi-shard test: ${shardUrls.length} shards × ${perShardN} = ${totalTarget} connections`, ); -console.log(`Hold: ${holdSec}s Ramp: ${rampPerSec}/s per shard\n`); -shardUrls.forEach((u, i) => console.log(` shard-${i + 1}: ${u}`)); -console.log(""); +console.log(`Target: ${target} Hold: ${holdSec}s Ramp: ${rampPerSec}/s per shard\n`); + +console.log(`Health sweep: ${shardUrls.length} shard(s)...`); +const health = await checkShardHealth(shardUrls); +const bad = health.filter((h) => !h.ok); +for (const h of bad) console.error(` ✗ ${h.url}: ${h.detail}`); +if (bad.length > 0) { + console.error( + `${bad.length}/${shardUrls.length} shard(s) unhealthy. Fix or drop them from SHARDS before burning a run.`, + ); + process.exit(1); +} +console.log(` ✓ all ${shardUrls.length} shards healthy\n`); -// Per-shard hard timeout — bumps fetch's headers timeout, but also acts as -// an absolute ceiling so one hung shard can't block the whole report. -const SHARD_TIMEOUT_MS = parseInt(process.env.SHARD_TIMEOUT_MS || "600000", 10); +// Ramp + hold bounds the wall clock; pad generously for connection retries. +const shardTimeoutMs = parseInt( + process.env.SHARD_TIMEOUT_MS || + String((Math.ceil(perShardN / rampPerSec) + holdSec + 300) * 1000), + 10, +); -async function runShard(url: string, label: string): Promise { - const qs = new URLSearchParams({ - n: String(perShardN), - hold: String(holdSec), - ramp: String(rampPerSec), +const specs: ShardSpec[] = shardUrls.map((url, i) => ({ + url, + label: `shard-${i + 1}`, + endpoint, + query: { + n: perShardN, + hold: holdSec, + ramp: rampPerSec, stream, - shard: label, - }); - if (target === "anycable" && cableUrl) qs.set("cableUrl", cableUrl); - if (target === "socketio" && socketioServerUrl) qs.set("serverUrl", socketioServerUrl); - if (target === "uws" && uwsWsUrl) qs.set("wsUrl", uwsWsUrl); - const endpoint = - target === "socketio" - ? "bench-idle-socketio" - : target === "uws" - ? "bench-idle-uws" - : "bench-idle-anycable"; - - const ctrl = new AbortController(); - const t = setTimeout(() => ctrl.abort(), SHARD_TIMEOUT_MS); - try { - const res = await fetch(`${url}/${endpoint}?${qs.toString()}`, { - method: "POST", - signal: ctrl.signal, - }); - if (!res.ok) { - throw new Error(`${label} returned ${res.status} ${res.statusText}`); - } - const result = (await res.json()) as IdleResult; - // Stream this shard's outcome immediately so a later stall can't lose it. - console.log( - ` ✓ ${label}: connected=${result.connected} welcomed=${result.welcomed} subscribed=${result.subscribed} failed=${result.failed} ramp=${(result.rampElapsedMs / 1000).toFixed(1)}s` - ); - return result; - } catch (err) { - console.log( - ` ✗ ${label}: ${err instanceof Error ? err.message : String(err)}` - ); - throw err; - } finally { - clearTimeout(t); - } -} + shard: `shard-${i + 1}`, + ...targetQuery, + }, +})); const startedAt = new Date(); const startedAtIso = startedAt.toISOString(); console.log(`Test started at ${startedAtIso}\n`); -// Use allSettled: a single shard's HTTP failure (502, network glitch, etc.) -// shouldn't lose the other shards' results. We report partial-success and -// keep going so the metrics chart still has data. -const shardPromises = shardUrls.map((u, i) => runShard(u, `shard-${i + 1}`)); -const settled = await Promise.allSettled(shardPromises); +const outcomes = await runShards(specs, { + pollIntervalMs: 5000, + pollLogLines: 10, + printProgress: true, + shardTimeoutMs, +}); const endedAt = new Date(); const endedAtIso = endedAt.toISOString(); @@ -141,11 +129,11 @@ const endedAtIso = endedAt.toISOString(); // Aggregate const results: IdleResult[] = []; -const errors: { idx: number; reason: string }[] = []; -settled.forEach((s, i) => { - if (s.status === "fulfilled") results.push(s.value); - else errors.push({ idx: i + 1, reason: String(s.reason).slice(0, 200) }); -}); +const errors: { label: string; reason: string }[] = []; +for (const o of outcomes) { + if (o.status === "done" && o.result) results.push(o.result); + else errors.push({ label: o.spec.label, reason: (o.error || "unknown").slice(0, 200) }); +} const totals = results.reduce( (acc, r) => ({ @@ -154,13 +142,12 @@ const totals = results.reduce( subscribed: acc.subscribed + r.subscribed, failed: acc.failed + r.failed, }), - { connected: 0, welcomed: 0, subscribed: 0, failed: 0 } + { connected: 0, welcomed: 0, subscribed: 0, failed: 0 }, ); -// Per-shard outcomes were already streamed via runShard; no need to repeat. if (errors.length > 0) { console.log( - `\n${errors.length} shard(s) errored or timed out — totals below cover the ${results.length} surviving shard(s).` + `\n${errors.length} shard(s) errored or timed out — totals below cover the ${results.length} surviving shard(s).`, ); } @@ -171,6 +158,25 @@ console.log(` Subscribed: ${totals.subscribed.toLocaleString()}`); console.log(` Failed: ${totals.failed.toLocaleString()}`); console.log(` Test window: ${startedAtIso} → ${endedAtIso}`); +// Validity: the uniform-shard-ceiling signature. Every shard freezing at +// the same count below PER_SHARD_N is the load generator's ephemeral-port +// or event-loop wall (the "exactly 12,002 per shard" bug class), never a +// server ceiling. Name the stop condition or re-run with more shards. +let generatorLimited = false; +if (results.length >= 2) { + const connected = results.map((r) => r.connected); + const allEqual = connected.every((c) => Math.abs(c - connected[0]) <= 5); + if (allEqual && connected[0] < perShardN * 0.99) { + generatorLimited = true; + console.log( + `\n[FATAL] uniform-shard-ceiling: every shard stopped at ~${connected[0]} of ${perShardN} requested.`, + ); + console.log( + ` This is the load generator's wall, not the server's. Add shards or lower PER_SHARD_N; do not publish this as a capacity number.`, + ); + } +} + // ------------------------------------------------------------------------- // Optional: Railway metrics + chart @@ -180,9 +186,9 @@ const serviceName = process.env.SERVICE_NAME || "anycable-go"; if (!projectId || !serviceId) { console.log( - "\n(set PROJECT_ID and SERVICE_ID to chart memory/CPU on anycable-go)" + "\n(set PROJECT_ID and SERVICE_ID to chart memory/CPU on the target)", ); - process.exit(0); + process.exit(generatorLimited ? 2 : 0); } const token = readRailwayToken(); @@ -193,7 +199,7 @@ const windowStart = new Date(startedAt.getTime() - padMs).toISOString(); const windowEnd = new Date(endedAt.getTime() + padMs).toISOString(); console.log( - `\nFetching Railway metrics for ${serviceName} over [${windowStart}, ${windowEnd}]...` + `\nFetching Railway metrics for ${serviceName} over [${windowStart}, ${windowEnd}]...`, ); const [memPoints, cpuPoints] = await Promise.all([ @@ -217,7 +223,7 @@ const [memPoints, cpuPoints] = await Promise.all([ if (memPoints.length === 0 && cpuPoints.length === 0) { console.log("(no metrics returned — wrong service id, or window too short?)"); - process.exit(0); + process.exit(generatorLimited ? 2 : 0); } // Convert to seconds-since-test-start. @@ -226,7 +232,7 @@ const memSeries = memPoints.map((p) => ({ tSec: p.ts - startUnix, value: p.value const cpuSeries = cpuPoints.map((p) => ({ tSec: p.ts - startUnix, value: p.value })); console.log( - `\n=== ${serviceName} during the test === (n=${memPoints.length} samples)\n` + `\n=== ${serviceName} during the test === (n=${memPoints.length} samples)\n`, ); console.log(chart({ title: `Memory`, points: memSeries, height: 12, width: 60, yUnit: "MB" })); @@ -239,6 +245,15 @@ const cpuValues = cpuSeries.map((p) => p.value).sort((a, b) => a - b); console.log(`\nMemory: peak=${percentile(memValues, 100).toFixed(0)} MB p95=${percentile(memValues, 95).toFixed(0)} MB avg=${(memValues.reduce((s, n) => s + n, 0) / Math.max(1, memValues.length)).toFixed(0)} MB`); console.log(`CPU: peak=${percentile(cpuValues, 100).toFixed(2)} % p95=${percentile(cpuValues, 95).toFixed(2)} % avg=${(cpuValues.reduce((s, n) => s + n, 0) / Math.max(1, cpuValues.length)).toFixed(2)} %`); +// RAM per connection while held: the metric that stays valid even when the +// fleet caps below the target scale (matched-scale efficiency). +if (totals.connected > 0 && memValues.length > 0) { + const peakMb = percentile(memValues, 100); + console.log( + `RAM/conn (peak): ${((peakMb * 1024) / totals.connected).toFixed(1)} KB across ${totals.connected.toLocaleString()} connections`, + ); +} + // CSV: tSec, mem_mb, cpu_pct (joined on closest sample timestamp). const csvPath = resultPath(`idle-multi-${startedAt.toISOString().replace(/[:.]/g, "-")}.csv`); const lines = ["t_sec,memory_mb,cpu_pct"]; @@ -252,3 +267,5 @@ for (const t of allTs) { } writeFileSync(csvPath, lines.join("\n") + "\n"); console.log(`\nWrote time-series CSV: ${csvPath}`); + +process.exit(generatorLimited ? 2 : 0); diff --git a/backend/src/bench/jitter-multi.ts b/backend/src/bench/jitter-multi.ts index 7e3dec8..4e45643 100644 --- a/backend/src/bench/jitter-multi.ts +++ b/backend/src/bench/jitter-multi.ts @@ -6,54 +6,34 @@ // per-shard downsampled latency samples (lib/stats.ts:mergeJitterResults) // so the reported p50/p95/p99 reflect the true union distribution. // -// Solves the single-bench-runner ~50K ceiling: 4 shards × 25K = 100K subs -// without saturating any single event loop. +// Solves the single-bench-runner saturation ceiling. Keep per-shard N near +// 250 for latency-accurate runs; a loaded runner inflates latency AND +// deflates delivery. // // Usage: // SHARDS=https://br-1.up.railway.app,https://br-2.up.railway.app,... \ -// TOTAL=100000 RAMP_PER_SEC=200 \ +// TOTAL=10000 RAMP_PER_SEC=200 \ // PROTOCOL=anycable \ # or socketio | socketio-csr | uws // TOTAL_MESSAGES=120 INTERVAL_MS=500 \ // JITTER_INTERVAL=15 JITTER_DURATION=1000 \ // SAMPLES_CAP=5000 \ # per shard; merged ~k× that // tsx src/bench/jitter-multi.ts // -// Per-protocol targets are picked up from the same env vars as the -// single-shard drivers (CABLE_URL, SERVER_URL, UWS_WS_URL, UWS_HTTP_URL) -// and forwarded to each shard as query params. +// Target overrides (CABLE_URL, BROADCAST_URL, CHANNEL, AC_PROTOCOL, +// RECONNECT_MODE, RECONNECT_BASE_MS, CLIENT_LIB, SERVER_URL, UWS_WS_URL, +// UWS_HTTP_URL) are forwarded via lib/core/multi-shard.ts — one mapping +// for every multi-shard driver. -import { writeFileSync } from "node:fs"; -import { Agent, setGlobalDispatcher } from "undici"; - -import { runShards, type ShardSpec } from "../lib/core/shard-coordinator.js"; import { - formatHumanReport, - mergeJitterResults, - type JitterResult, -} from "../lib/core/stats.js"; - -// The coordinator does short polls, but the cumulative wait is bounded by -// the longest shard. Push the default fetch timeouts up so the enqueue + -// poll fetches never give up before Railway does. -setGlobalDispatcher( - new Agent({ headersTimeout: 30 * 60 * 1000, bodyTimeout: 30 * 60 * 1000 }), -); + ENDPOINTS, + parseProtocol, + parseShardUrls, + protocolTargetQuery, + runMultiShard, +} from "../lib/core/multi-shard.js"; -const shardsCsv = process.env.SHARDS; -if (!shardsCsv) { - console.error( - "SHARDS env var required (comma-separated bench-runner base URLs)", - ); - process.exit(1); -} -const shardUrls = shardsCsv - .split(",") - .map((s) => s.trim()) - .filter(Boolean); -if (shardUrls.length === 0) { - console.error("SHARDS must contain at least one URL"); - process.exit(1); -} +const shardUrls = parseShardUrls(); +const protocol = parseProtocol(); const totalClients = parseInt(process.env.TOTAL || "10000", 10); const perShardN = Math.ceil(totalClients / shardUrls.length); @@ -64,60 +44,25 @@ const durationSec = parseInt(process.env.DURATION || "160", 10); const jitterIntervalSec = parseInt(process.env.JITTER_INTERVAL || "15", 10); const jitterDurationMs = parseInt(process.env.JITTER_DURATION || "1000", 10); const samplesCap = parseInt(process.env.SAMPLES_CAP || "5000", 10); -const protocol = (process.env.PROTOCOL || "anycable").toLowerCase(); -const PROTOCOL_TO_ENDPOINT: Record = { - anycable: "bench-jitter-anycable", - socketio: "bench-jitter-socketio", - "socketio-csr": "bench-jitter-socketio-csr", - uws: "bench-jitter-uws", -}; -const endpoint = PROTOCOL_TO_ENDPOINT[protocol]; -if (!endpoint) { - console.error( - `PROTOCOL must be one of: ${Object.keys(PROTOCOL_TO_ENDPOINT).join(", ")} (got "${protocol}")`, - ); - process.exit(1); -} - -// Per-protocol target overrides forwarded to each shard. The bench-runner -// already supports these via its own query-string parsers. -const protocolQuery: Record = {}; -if (protocol === "anycable") { - if (process.env.CABLE_URL) protocolQuery.cableUrl = process.env.CABLE_URL; - if (process.env.BROADCAST_URL) - protocolQuery.broadcastUrl = process.env.BROADCAST_URL; -} -if (protocol === "socketio" || protocol === "socketio-csr") { - if (process.env.SERVER_URL) protocolQuery.serverUrl = process.env.SERVER_URL; -} -if (protocol === "uws") { - if (process.env.UWS_WS_URL) protocolQuery.wsUrl = process.env.UWS_WS_URL; - if (process.env.UWS_HTTP_URL) protocolQuery.httpUrl = process.env.UWS_HTTP_URL; -} +const targetQuery = protocolTargetQuery(protocol); console.log( `Multi-shard jitter: ${shardUrls.length} shards × ${perShardN} = ${shardUrls.length * perShardN} clients (target: ${totalClients})`, ); console.log( - `Protocol: ${protocol} Endpoint: /${endpoint} Stream prefix: jitter-multi-${Date.now()}`, -); -console.log( - `Per-shard: msgs=${totalMessages} intervalMs=${intervalMs} ramp=${rampPerSec}/s duration=${durationSec}s`, + `Protocol: ${protocol} Per-shard: msgs=${totalMessages} intervalMs=${intervalMs} ramp=${rampPerSec}/s duration=${durationSec}s Samples: ${samplesCap}/shard`, ); -console.log(`Samples: ${samplesCap} per shard`); console.log(""); -// Unique stream per shard so each shard's publisher fans out only to its -// own subscribers. If we shared a stream across shards, every shard's -// clients would receive all shards' publishes and the deliveryRate math -// would be wrong. -const runStamp = Date.now(); -const shardSpecs: ShardSpec[] = shardUrls.map((url, i) => ({ - url, - label: `shard-${i + 1}`, - endpoint, - query: { +const run = await runMultiShard({ + testType: "jitter", + protocol, + endpoint: ENDPOINTS.jitter[protocol], + shardUrls, + // Unique stream per shard so each shard's publisher fans out only to its + // own subscribers; a shared stream would break the deliveryRate math. + shardQuery: (i, runStamp) => ({ n: perShardN, msgs: totalMessages, interval: intervalMs, @@ -127,90 +72,21 @@ const shardSpecs: ShardSpec[] = shardUrls.map((url, i) => ({ jitterMs: jitterDurationMs, samplesCap, stream: `jitter-multi-${runStamp}-s${i + 1}`, - ...protocolQuery, + ...targetQuery, + }), + validity: { perShardN, expectedDurationSec: durationSec }, + meta: { + totalClients, + perShardN, + totalMessages, + intervalMs, + rampPerSec, + durationSec, + jitterIntervalSec, + jitterDurationMs, + samplesCap, + targetQuery, }, -})); - -const startedAt = new Date(); -const outcomes = await runShards(shardSpecs, { - pollIntervalMs: 5000, - pollLogLines: 30, - printProgress: true, }); -const endedAt = new Date(); - -const successes = outcomes.filter( - (o): o is typeof o & { result: JitterResult } => - o.status === "done" && o.result !== undefined, -); -const failures = outcomes.filter((o) => o.status !== "done"); - -console.log(""); -console.log(`=== Per-shard === (${successes.length} succeeded / ${outcomes.length})`); -for (const o of outcomes) { - if (o.status === "done" && o.result) { - const r = o.result; - console.log( - ` ${o.spec.label}: clients=${r.clients} delivery=${r.deliveryRatePct}% p99=${r.latencyOverMinMs.p99}ms (${(o.durationMs / 1000).toFixed(1)}s)`, - ); - } else { - console.log(` ${o.spec.label}: FAILED — ${o.error || "unknown"}`); - } -} - -if (failures.length > 0) { - console.log(""); - console.log(`!! ${failures.length} shard(s) failed; reported merge covers the remaining ${successes.length}.`); -} - -if (successes.length === 0) { - console.error("All shards failed; no merged result to report."); - process.exit(1); -} - -const merged = mergeJitterResults( - `${protocol}-multi-${shardUrls.length}x${perShardN}`, - successes.map((o) => o.result), -); - -console.log(formatHumanReport(`Merged (${shardUrls.length} shards)`, merged)); - -// Drop a JSON file for the manifest tooling (#23) to pick up later. -const outPath = `jitter-multi-${protocol}-${shardUrls.length}x${perShardN}-${startedAt.toISOString().replace(/[:.]/g, "-")}.json`; -writeFileSync( - outPath, - JSON.stringify( - { - protocol, - shards: shardUrls.length, - perShardN, - totalClients: shardUrls.length * perShardN, - startedAt: startedAt.toISOString(), - endedAt: endedAt.toISOString(), - params: { - totalMessages, - intervalMs, - rampPerSec, - durationSec, - jitterIntervalSec, - jitterDurationMs, - samplesCap, - }, - perShard: outcomes.map((o) => ({ - label: o.spec.label, - url: o.spec.url, - status: o.status, - jobId: o.jobId, - durationMs: o.durationMs, - result: o.result, - error: o.error, - })), - merged, - }, - null, - 2, - ), -); -console.log(`\nWrote merged JSON: ${outPath}`); -process.exit(merged.lostDeliveries > 0 ? 1 : 0); +process.exit(run.exitCode); diff --git a/backend/src/bench/preflight.ts b/backend/src/bench/preflight.ts new file mode 100644 index 0000000..efbac88 --- /dev/null +++ b/backend/src/bench/preflight.ts @@ -0,0 +1,305 @@ +// Pre-run fairness and fleet audit. Run before ANY paid benchmark window. +// +// TARGETS=rails-actioncable,rails-anycable,anycable-go-rails \ +// SHARDS=https://bench-runner-2-production.up.railway.app,... \ +// npm run bench:preflight +// +// npm run bench:fleet # inventory/diff only (no SHARDS/TARGETS needed) +// +// Encodes the checks whose absence cost full re-runs in past campaigns: +// stale runner images (locally-validated code that never shipped), secret +// drift across the fleet (silent publisher 401s), deploy churn during a +// window (contaminated numbers), dead targets, and misconfigured env. +// What it cannot verify mechanically it prints as a manual checklist +// (box sizes, worker counts from boot logs). +// +// Exit codes: 0 all checks passed, 1 at least one FAIL. + +import { execSync } from "node:child_process"; +import { existsSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { checkShardHealth } from "../lib/core/multi-shard.js"; +import { + getServiceVariables, + isChurning, + isLive, + listServiceStates, + readManifest, + readRailwayToken, + resolveProject, + sha12, + type ServiceState, +} from "../lib/core/railway-fleet.js"; + +const mode = process.argv[2] === "status" ? "status" : "preflight"; + +const here = dirname(fileURLToPath(import.meta.url)); +const manifestPath = join(here, "..", "..", "..", "fleet-manifest.json"); +if (!existsSync(manifestPath)) { + console.error(`fleet-manifest.json not found at ${manifestPath}`); + process.exit(1); +} +const manifest = readManifest(manifestPath); +const runnerRe = new RegExp(manifest.runnerPattern); + +let failures = 0; +let warnings = 0; +const fail = (msg: string) => { + failures++; + console.log(` [FAIL] ${msg}`); +}; +const warn = (msg: string) => { + warnings++; + console.log(` [warn] ${msg}`); +}; +const pass = (msg: string) => console.log(` [ ok ] ${msg}`); + +// --------------------------------------------------------------------------- +// 1. Railway auth + project resolution + +console.log("== Railway auth =="); +let token: string; +try { + token = readRailwayToken(); +} catch (e) { + fail(`cannot read Railway token: ${(e as Error).message}. Run 'railway login'.`); + process.exit(1); +} +let resolved; +try { + resolved = await resolveProject(manifest, token); + pass(`project "${manifest.project}" / env "${manifest.environment}" resolved`); +} catch (e) { + fail((e as Error).message); + process.exit(1); +} + +// --------------------------------------------------------------------------- +// 2. Fleet inventory + churn + +console.log("\n== Fleet inventory =="); +let states: ServiceState[]; +try { + states = await listServiceStates(resolved, token); +} catch (e) { + fail((e as Error).message); + process.exit(1); +} +const live = states.filter(isLive); +const churning = states.filter(isChurning); +const liveRunners = live.filter((s) => runnerRe.test(s.serviceName)); +const liveOther = live.filter((s) => !runnerRe.test(s.serviceName)); +console.log( + ` ${states.length} services; ${live.length} live (${liveRunners.length} runners, ${liveOther.length} other); ${states.length - live.length} down`, +); +for (const s of liveOther) { + console.log(` live: ${s.serviceName} (${s.deployment?.status})`); +} +if (churning.length > 0) { + fail( + `deploy churn in progress: ${churning.map((s) => s.serviceName).join(", ")}. Never measure during churn; wait for it to settle.`, + ); +} else { + pass("no deploy churn"); +} + +if (mode === "status") { + console.log( + `\nManifest expects everything torn down between campaigns; ${live.length} live service(s) are billing right now.`, + ); + process.exit(failures > 0 ? 1 : 0); +} + +// --------------------------------------------------------------------------- +// 3. Targets + +const targetNames = (process.env.TARGETS || "") + .split(",") + .map((s) => s.trim()) + .filter(Boolean); + +if (targetNames.length > 0) { + console.log("\n== Targets =="); + for (const name of targetNames) { + const state = states.find((s) => s.serviceName === name); + if (!state) { + fail(`target "${name}" does not exist in the project`); + continue; + } + if (!isLive(state)) { + fail( + `target "${name}" has no live deployment (status: ${state.deployment?.status ?? "none"})`, + ); + continue; + } + pass(`${name} live (deployed ${state.deployment?.createdAt})`); + const expect = manifest.targets[name]; + if (expect) { + try { + const vars = await getServiceVariables(resolved, state.serviceId, token); + for (const key of expect.expectEnv) { + if (!(key in vars)) fail(`${name}: expected env var ${key} is not set`); + else pass(`${name}: ${key} set`); + } + for (const [key, forbidden] of Object.entries(expect.forbidEnv ?? {})) { + if (vars[key] === forbidden) { + fail(`${name}: ${key}=${forbidden} makes runs silently invalid (see manifest note)`); + } else { + pass(`${name}: ${key} is not ${forbidden}`); + } + } + // Heap cap sanity: a NODE_OPTIONS max-old-space-size left over from a + // smaller box turns reconnect storms into kernel OOM-kill loops. + if (vars.NODE_OPTIONS && /max-old-space-size/.test(vars.NODE_OPTIONS)) { + warn( + `${name}: NODE_OPTIONS pins the heap (${vars.NODE_OPTIONS}); confirm it fits the current container size`, + ); + } + } catch (e) { + warn(`${name}: could not read variables (${(e as Error).message})`); + } + if (expect.notes) console.log(` note: ${expect.notes}`); + } else { + warn(`${name}: not in fleet-manifest.json targets; add it with expectEnv + notes`); + } + } +} + +// --------------------------------------------------------------------------- +// 4. Shards: HTTP health + secrets parity + image freshness + +const shardUrls = (process.env.SHARDS || "") + .split(",") + .map((s) => s.trim()) + .filter(Boolean); + +if (shardUrls.length > 0) { + console.log(`\n== Shards (${shardUrls.length}) ==`); + + // Map public URLs back to service names: bench-runner-7-production.up.railway.app + // → bench-runner-7 (and bench-runner-production → bench-runner). + const shardServiceNames = shardUrls.map((u) => { + const host = u.replace(/^https?:\/\//, "").split("/")[0]; + return host.replace(/-production\.up\.railway\.app.*$/, ""); + }); + const excluded = shardServiceNames.filter((n) => + manifest.excludedRunners.includes(n), + ); + if (excluded.length > 0) { + fail( + `SHARDS includes excluded runner(s): ${excluded.join(", ")} (see fleet-manifest.json; bench-runner-81 is the Centrifugo target)`, + ); + } + + // HTTP health + auth + async-API probe. + const health = await checkShardHealth(shardUrls); + const badHealth = health.filter((h) => !h.ok); + if (badHealth.length === 0) pass(`all ${shardUrls.length} shards healthy over HTTP`); + for (const h of badHealth) fail(`${h.url}: ${h.detail}`); + + // Image freshness: every shard's deployment must postdate the last local + // commit touching backend/. A stale runner runs old driver code and + // produces confusing half-failures, never an error. + let headTime: Date | null = null; + try { + headTime = new Date( + execSync("git log -1 --format=%cI -- .", { cwd: join(here, "..", "..") }) + .toString() + .trim(), + ); + // Uncommitted driver/runner changes are invisible to the timestamp + // check: the deployed image can look "fresh" while missing them. + const dirty = execSync("git status --porcelain -- src", { + cwd: join(here, "..", ".."), + }) + .toString() + .trim(); + if (dirty) { + warn( + "backend/src has uncommitted changes; the freshness check only sees commits. Commit and redeploy before trusting the fleet.", + ); + } + } catch { + warn("could not read git HEAD time; skipping image freshness check"); + } + if (headTime) { + const stale: string[] = []; + for (const name of shardServiceNames) { + const state = states.find((s) => s.serviceName === name); + if (!state || !isLive(state)) { + fail(`shard service "${name}" has no live deployment`); + continue; + } + if (state.deployment && new Date(state.deployment.createdAt) < headTime) { + stale.push(name); + } + } + if (stale.length > 0) { + fail( + `${stale.length} shard(s) run an image OLDER than the last backend commit: ${stale.join(", ")}.\n` + + ` Redeploy them (concurrency <= 4, retry stragglers):\n` + + ` for s in ${stale.join(" ")}; do railway up --service "$s" --ci --detach; done`, + ); + } else { + pass("every shard image postdates the last backend commit"); + } + } + + // Secrets parity: sharedSecrets must hash identically on every shard + // (and match the targets where applicable). Values never printed. + console.log(" secrets parity (sha12, values never printed):"); + const hashesBySecret: Record> = {}; + for (const name of shardServiceNames) { + const state = states.find((s) => s.serviceName === name); + if (!state) continue; + try { + const vars = await getServiceVariables(resolved, state.serviceId, token); + for (const secret of manifest.sharedSecrets) { + const h = vars[secret] ? sha12(vars[secret]) : "(unset)"; + hashesBySecret[secret] ??= new Map(); + const list = hashesBySecret[secret].get(h) ?? []; + list.push(name); + hashesBySecret[secret].set(h, list); + } + } catch (e) { + warn(`${name}: could not read variables (${(e as Error).message})`); + } + } + for (const secret of manifest.sharedSecrets) { + const groups = hashesBySecret[secret]; + if (!groups) continue; + if (groups.size === 1 && !groups.has("(unset)")) { + pass(`${secret}: identical on all ${shardServiceNames.length} shards`); + } else { + for (const [h, names] of groups) { + fail( + `${secret}: ${h === "(unset)" ? "NOT SET" : `hash ${h}`} on ${names.length} shard(s): ${names.slice(0, 8).join(", ")}${names.length > 8 ? ", ..." : ""}`, + ); + } + console.log( + ` Secret drift causes silent publisher 401s (the uniform-65%-delivery bug class). Fix before running.`, + ); + } + } +} + +// --------------------------------------------------------------------------- +// 5. What this script cannot verify + +console.log(`\n== Manual checks (cannot be verified via API) == + - Container sizes EQUAL across all compared targets, set explicitly in the + dashboard. Results that swap after a resize invalidate everything pre-resize. + - Worker/process counts from BOOT LOGS, never env vars: + railway logs --service rails-actioncable | grep -i "cluster mode\\|Worker" + (stock puma.rb silently ignores WEB_CONCURRENCY without a workers directive) + - One continuous window for every compared row; no deploys of anything + (including more shards) once measurement starts. + - Teardown plan for the end of the window (deploymentRemove per service, + verify public domains return 404; limits changes alone keep billing).`); + +console.log( + `\n${failures === 0 ? "PREFLIGHT PASSED" : `PREFLIGHT FAILED (${failures} failure(s))`}${warnings > 0 ? `, ${warnings} warning(s)` : ""}`, +); +process.exit(failures > 0 ? 1 : 0); diff --git a/backend/src/bench/rebaseline.ts b/backend/src/bench/rebaseline.ts index 5f9b54d..fa4bd27 100644 --- a/backend/src/bench/rebaseline.ts +++ b/backend/src/bench/rebaseline.ts @@ -30,6 +30,12 @@ import { tests, type TestSpec } from "./tests-manifest.js"; import { runShards, type ShardSpec } from "../lib/core/shard-coordinator.js"; import { fetchMetric, readRailwayToken } from "../lib/core/railway-api.js"; import { benchRunnerFetch } from "../lib/core/bench-runner-client.js"; +import { mergeJitterResults, type JitterResult } from "../lib/core/stats.js"; +import { + formatValidityReport, + validateResult, + validateShardSet, +} from "../lib/core/validity.js"; // Railway project that hosts the bench targets. Hardcoded because it's // stable across runs; can override with PROJECT_ID for a different env. @@ -333,6 +339,74 @@ async function runMultiShard(spec: TestSpec): Promise { } // Wraps runMultiShard to also fetch + attach Railway metrics for the target. +// Fan a jitter/latency test across `spec.numShards` replicas and merge the +// JitterResults properly: union percentiles from per-shard downsampled +// samples (mergeJitterResults), never per-shard averages. Runs the validity +// checks so a run that measured the rig flags itself in the report. +async function runMultiJitter(spec: TestSpec): Promise { + if (!spec.numShards || !spec.perShardN) { + throw new Error(`${spec.id}: multi-jitter mode needs numShards + perShardN`); + } + if (benchRunnerUrls.length < spec.numShards) { + throw new Error( + `${spec.id}: needs ${spec.numShards} shards but BENCH_RUNNER_URLS only has ${benchRunnerUrls.length}`, + ); + } + const runStamp = Date.now(); + const shards: ShardSpec[] = benchRunnerUrls + .slice(0, spec.numShards) + .map((url, i) => ({ + url, + label: `s${i + 1}`, + endpoint: spec.endpoint, + query: { + ...Object.fromEntries( + Object.entries(spec.params).map(([k, v]) => [k, String(v)]), + ), + // After the params spread so a spec's n/stream can never override + // the sharding: per-shard N, merge-ready samples, unique stream so + // each shard's publisher fans out only to its own subscribers. + n: spec.perShardN!, + samplesCap: Number(spec.params.samplesCap ?? 5000), + stream: `${spec.id}-${runStamp}-s${i + 1}`, + }, + })); + + const outcomes = await runShards(shards, { + pollIntervalMs: 10_000, + printProgress: false, + shardTimeoutMs: 15 * 60 * 1000, + }); + + const successes = outcomes + .filter((o) => o.status === "done" && o.result) + .map((o) => o.result!) as JitterResult[]; + const failed = outcomes.length - successes.length; + if (failed > 0) { + console.log( + ` ${c.yellow}${failed} of ${outcomes.length} shard(s) failed; merge covers the rest — treat as a partial run${c.reset}`, + ); + } + if (successes.length === 0) { + throw new Error(`${spec.id}: all ${outcomes.length} shards failed`); + } + + const merged = mergeJitterResults(spec.id, successes); + + const durationSec = Number(spec.params.duration); + const flags = [ + ...validateShardSet(successes, { perShardN: spec.perShardN }), + ...validateResult(merged, { + perShardN: spec.perShardN, + expectedDurationSec: Number.isFinite(durationSec) ? durationSec : undefined, + }), + ]; + if (flags.length > 0) { + console.log(formatValidityReport(flags)); + } + return merged; +} + async function runMultiShardWithMetrics( spec: TestSpec, ): Promise { @@ -472,6 +546,9 @@ async function runTest(spec: TestSpec, baseUrl: string): Promise { if (spec.mode === "multi-shard") { return runMultiShardWithMetrics(spec); } + if (spec.mode === "multi-jitter") { + return runMultiJitter(spec); + } if (spec.mode === "avalanche") { return runAvalancheWithRedeploy(spec, baseUrl); } diff --git a/backend/src/bench/tests-manifest.ts b/backend/src/bench/tests-manifest.ts index ad89784..e027977 100644 --- a/backend/src/bench/tests-manifest.ts +++ b/backend/src/bench/tests-manifest.ts @@ -43,14 +43,20 @@ export interface TestSpec { // - "sync" blocks on the response. <5 min tests only. // - "async" enqueues, polls /jobs/:id. For tests that may run >5 min. // - "multi-shard" fans out across N bench-runner replicas via the - // shard-coordinator; rebaseline merges per-shard results. - // Set numShards + perShardN. Idle capacity tests use this. + // shard-coordinator and SUMS idle-style counts + // (connected/welcomed/subscribed/failed). Set numShards + + // perShardN. Idle capacity tests use this. + // - "multi-jitter" fans out a jitter/latency test across N replicas and + // merges JitterResults properly (union percentiles via + // latencySamplesSorted), running the validity checks. + // Set numShards + perShardN; keep perShardN near 250 — + // a loaded runner inflates latency and deflates delivery. // - "avalanche" async test where the runner triggers `railway service // redeploy` mid-flight to simulate the in-process WS // layer restarting under N held connections. Set // redeployServiceName + the bench-runner endpoint's // prearmSec param. - mode: "sync" | "async" | "multi-shard" | "avalanche"; + mode: "sync" | "async" | "multi-shard" | "multi-jitter" | "avalanche"; params: Record; // Page baseline values, keyed by dotted paths into the result JSON. baseline: Record; @@ -101,13 +107,43 @@ const TARGETS = { // See docs/socketioxide-comparison.md for the open question to the // library author. socketioxide: "http://socketioxide-server.railway.internal:3000", + + // Rails broadcasting comparison (AnyCable vs Action Cable vs Solid Cable). + // One Rails app (cable-bench/), three deployments selected by BENCH_MODE. + // Action Cable / Solid Cable terminate WebSockets in Puma and expose the + // app's POST /_bench/broadcast publish endpoint; the bench-runner reuses the + // anycable jitter/idle/avalanche endpoints with ?channel=BenchmarkChannel and + // ?acProtocol=actioncable-v1-json. AnyCable terminates in a separate + // anycable-go gateway (RPC -> the Rails app) and publishes via the gateway's + // /_broadcast, exactly like the standalone AnyCable target, but over the + // extended protocol and a real BenchmarkChannel. + railsSolidCable: "ws://rails-solidcable.railway.internal:3000/cable", + railsSolidCableBroadcast: "http://rails-solidcable.railway.internal:3000/_bench/broadcast", + railsActionCable: "ws://rails-actioncable.railway.internal:3000/cable", + railsActionCableBroadcast: "http://rails-actioncable.railway.internal:3000/_bench/broadcast", + railsAnyCable: "ws://anycable-go-rails.railway.internal:8080/cable", + railsAnyCableBroadcast: "http://anycable-go-rails.railway.internal:8080/_broadcast", + // AsyncCable: standard Action Cable wire protocol, served in-process by + // Falcon (async/fibers) instead of Puma. Same /cable + /_bench/broadcast + // surface as the other in-process Rails targets. + railsAsyncCable: "ws://rails-asynccable.railway.internal:3000/cable", + railsAsyncCableBroadcast: "http://rails-asynccable.railway.internal:3000/_bench/broadcast", }; +// Action Cable subscribe presets. BenchmarkChannel is the channel the Rails +// app exposes (cable-bench/app/channels/benchmark_channel.rb). Vanilla Action +// Cable / Solid Cable speak the base protocol; AnyCable the extended one. +const RAILS_BASE = { channel: "BenchmarkChannel", acProtocol: "actioncable-v1-json" }; +const RAILS_EXT = { channel: "BenchmarkChannel", acProtocol: "actioncable-v1-ext-json" }; + // Common knobs reused across tests. Keep these explicit so the manifest // is self-documenting; copy-paste is fine when a test deviates. const LATENCY_1K = { msgs: 100, interval: 500, ramp: 100, duration: 90, jitter: 999999 }; const LATENCY_10K = { msgs: 100, interval: 500, ramp: 200, duration: 130, jitter: 999999 }; const JITTER_10K = { msgs: 120, interval: 500, ramp: 200, duration: 160, jitter: 15, jitterMs: 1000 }; +// Rails jitter runs enforce a standard 2 s outage (disconnect, wait, connect) +// so every client library faces the same disruption regardless of backoff. +const RAILS_JITTER_5K = { msgs: 120, interval: 500, ramp: 200, duration: 160, jitter: 15, jitterMs: 2000 }; const WHISPERS_1K = { rooms: 10, ramp: 100, interval: 500, duration: 30, payload: 64 }; const THROUGHPUT_10K_1M = { n: 10000, total: 100, intervalMs: 10, ramp: 200, drain: 30, publisher: "pool", publisherConcurrency: 16 }; @@ -629,4 +665,288 @@ export const tests: TestSpec[] = [ baseline: {}, driftThresholdPct: 100, }, + + // =========================================================================== + // Rails broadcasting: AnyCable vs Action Cable vs Solid Cable vs Async::Cable + // + // One Rails app, four cable backends. All speak Action Cable at the app + // level (same BenchmarkChannel), but: Solid Cable and Action Cable terminate + // WebSockets in Puma (in-process Ruby; Solid Cable also polls the DB), + // Async::Cable serves them from Falcon fibers, while AnyCable offloads them + // to anycable-go (Rails is only the gRPC backend) and speaks the extended + // protocol with delivery guarantees. Latency/jitter baselines mirror the + // numbers published on compare/rails-actioncable (2026-07-01 native-client + // window). Fairness preconditions the numbers depend on (verify via + // bench:preflight before trusting a red/green): WEB_CONCURRENCY=8 actually + // applied (boot logs, cluster mode), falcon --count matched, equal box + // sizes. Reuses the anycable bench-runner endpoints via ?channel + + // ?acProtocol; no new endpoints for latency/jitter. + // =========================================================================== + + // Latency (jitter-disabled roundtrip). SHARDED: single-runner runs at 5K + // produced the retracted 225 ms artifact (the runner's event loop, not the + // adapter); keep per-shard N at 250. Baselines are the numbers published + // on compare/rails-actioncable (2026-07-01 native-client window); the + // usual manifest caveat applies — a quieter window may come in lower. + // Shared-tenant swings are large, hence the wide thresholds. + { + id: "latency-rails-actioncable-1k", + description: "Roundtrip latency, Rails + Action Cable (Redis), 1K subs (4x250)", + category: "latency", + endpoint: "bench-jitter-anycable", + mode: "multi-jitter", + numShards: 4, + perShardN: 250, + params: { ...LATENCY_1K, ...RAILS_BASE, cableUrl: TARGETS.railsActionCable, broadcastUrl: TARGETS.railsActionCableBroadcast }, + baseline: { "latencyRawMs.p50": 11, "latencyRawMs.p99": 48, deliveryRatePct: 100 }, + driftThresholdPct: 50, + }, + { + id: "latency-rails-actioncable-5k", + description: "Roundtrip latency, Rails + Action Cable (Redis), 5K subs (20x250)", + category: "latency", + endpoint: "bench-jitter-anycable", + mode: "multi-jitter", + numShards: 20, + perShardN: 250, + params: { ...LATENCY_10K, ...RAILS_BASE, cableUrl: TARGETS.railsActionCable, broadcastUrl: TARGETS.railsActionCableBroadcast }, + baseline: { "latencyRawMs.p50": 11, "latencyRawMs.p99": 48, deliveryRatePct: 100 }, + driftThresholdPct: 50, + }, + { + id: "latency-rails-solidcable-1k", + description: "Roundtrip latency, Rails + Solid Cable, 1K subs (4x250)", + category: "latency", + endpoint: "bench-jitter-anycable", + mode: "multi-jitter", + numShards: 4, + perShardN: 250, + params: { ...LATENCY_1K, ...RAILS_BASE, cableUrl: TARGETS.railsSolidCable, broadcastUrl: TARGETS.railsSolidCableBroadcast }, + baseline: { "latencyRawMs.p50": 68, "latencyRawMs.p99": 143, deliveryRatePct: 100 }, + driftThresholdPct: 50, + }, + { + id: "latency-rails-solidcable-5k", + description: "Roundtrip latency, Rails + Solid Cable, 5K subs (20x250)", + category: "latency", + endpoint: "bench-jitter-anycable", + mode: "multi-jitter", + numShards: 20, + perShardN: 250, + params: { ...LATENCY_10K, ...RAILS_BASE, cableUrl: TARGETS.railsSolidCable, broadcastUrl: TARGETS.railsSolidCableBroadcast }, + baseline: { "latencyRawMs.p50": 88, "latencyRawMs.p99": 190, deliveryRatePct: 100 }, + driftThresholdPct: 50, + }, + { + id: "latency-rails-asynccable-1k", + description: "Roundtrip latency, Rails + Async::Cable (Falcon), 1K subs (4x250)", + category: "latency", + endpoint: "bench-jitter-anycable", + mode: "multi-jitter", + numShards: 4, + perShardN: 250, + params: { ...LATENCY_1K, ...RAILS_BASE, cableUrl: TARGETS.railsAsyncCable, broadcastUrl: TARGETS.railsAsyncCableBroadcast }, + baseline: { "latencyRawMs.p50": 15, "latencyRawMs.p99": 50, deliveryRatePct: 100 }, + driftThresholdPct: 50, + }, + { + id: "latency-rails-asynccable-5k", + description: "Roundtrip latency, Rails + Async::Cable (Falcon), 5K subs (20x250)", + category: "latency", + endpoint: "bench-jitter-anycable", + mode: "multi-jitter", + numShards: 20, + perShardN: 250, + params: { ...LATENCY_10K, ...RAILS_BASE, cableUrl: TARGETS.railsAsyncCable, broadcastUrl: TARGETS.railsAsyncCableBroadcast }, + baseline: { "latencyRawMs.p50": 15, "latencyRawMs.p99": 55, deliveryRatePct: 100 }, + driftThresholdPct: 50, + }, + { + id: "latency-rails-anycable-1k", + description: "Roundtrip latency, Rails + AnyCable (Go gateway), 1K subs (4x250)", + category: "latency", + endpoint: "bench-jitter-anycable", + mode: "multi-jitter", + numShards: 4, + perShardN: 250, + params: { ...LATENCY_1K, ...RAILS_EXT, cableUrl: TARGETS.railsAnyCable, broadcastUrl: TARGETS.railsAnyCableBroadcast }, + baseline: { "latencyRawMs.p50": 6, "latencyRawMs.p99": 22, deliveryRatePct: 100 }, + driftThresholdPct: 50, + }, + { + id: "latency-rails-anycable-5k", + description: "Roundtrip latency, Rails + AnyCable (Go gateway), 5K subs (20x250)", + category: "latency", + endpoint: "bench-jitter-anycable", + mode: "multi-jitter", + numShards: 20, + perShardN: 250, + params: { ...LATENCY_10K, ...RAILS_EXT, cableUrl: TARGETS.railsAnyCable, broadcastUrl: TARGETS.railsAnyCableBroadcast }, + baseline: { "latencyRawMs.p50": 6, "latencyRawMs.p99": 29, deliveryRatePct: 100 }, + driftThresholdPct: 50, + }, + + // Reliability under WiFi jitter (2 s enforced outage). NATIVE CLIENTS: + // the Action Cable family runs @rails/actioncable (clientLib=actioncable, + // poll-based reconnect, no resume) and AnyCable runs @anycable/core with + // its stock backoff — the client library is a first-order variable here + // (switching to native clients moved delivery 78% -> 53%). AnyCable's p99 + // tail is client reconnect backoff, never server replay; expect ~2 s. + { + id: "jitter-rails-actioncable-5k", + description: "Reliability under WiFi jitter, Rails + Action Cable (Redis), 5K (20x250, native client)", + category: "jitter", + endpoint: "bench-jitter-anycable", + mode: "multi-jitter", + numShards: 20, + perShardN: 250, + params: { ...RAILS_JITTER_5K, ...RAILS_BASE, clientLib: "actioncable", cableUrl: TARGETS.railsActionCable, broadcastUrl: TARGETS.railsActionCableBroadcast }, + baseline: { deliveryRatePct: 53, "latencyRawMs.p50": 12 }, + driftThresholdPct: 15, + }, + { + id: "jitter-rails-solidcable-5k", + description: "Reliability under WiFi jitter, Rails + Solid Cable, 5K (20x250, native client)", + category: "jitter", + endpoint: "bench-jitter-anycable", + mode: "multi-jitter", + numShards: 20, + perShardN: 250, + params: { ...RAILS_JITTER_5K, ...RAILS_BASE, clientLib: "actioncable", cableUrl: TARGETS.railsSolidCable, broadcastUrl: TARGETS.railsSolidCableBroadcast }, + baseline: { deliveryRatePct: 53, "latencyRawMs.p50": 68 }, + driftThresholdPct: 15, + }, + { + id: "jitter-rails-asynccable-5k", + description: "Reliability under WiFi jitter, Rails + Async::Cable (Falcon), 5K (20x250, native client)", + category: "jitter", + endpoint: "bench-jitter-anycable", + mode: "multi-jitter", + numShards: 20, + perShardN: 250, + params: { ...RAILS_JITTER_5K, ...RAILS_BASE, clientLib: "actioncable", cableUrl: TARGETS.railsAsyncCable, broadcastUrl: TARGETS.railsAsyncCableBroadcast }, + baseline: { deliveryRatePct: 53, "latencyRawMs.p50": 14 }, + driftThresholdPct: 15, + }, + { + id: "jitter-rails-anycable-5k", + description: "Reliability under WiFi jitter, Rails + AnyCable, 5K (20x250, stock @anycable/core backoff)", + category: "jitter", + endpoint: "bench-jitter-anycable", + mode: "multi-jitter", + numShards: 20, + perShardN: 250, + params: { ...RAILS_JITTER_5K, ...RAILS_EXT, cableUrl: TARGETS.railsAnyCable, broadcastUrl: TARGETS.railsAnyCableBroadcast }, + // p99 is the client's reconnect backoff window, wide threshold. + baseline: { deliveryRatePct: 100, "latencyRawMs.p50": 7, "latencyRawMs.p99": 2000 }, + driftThresholdPct: 60, + }, + + // Idle capacity. In-process Puma (Solid/Action Cable) tops out far below the + // Go gateway; targets are sized to find each ceiling (in-process ~200K probe, + // AnyCable 1M). Fill targetServiceId after deploy to attach Railway memory/CPU. + { + id: "idle-rails-solidcable", + description: "Idle connections held, Rails + Solid Cable", + category: "idle", + endpoint: "bench-idle-anycable", + mode: "multi-shard", + numShards: 13, + perShardN: 4000, + params: { hold: 120, ramp: 200, stream: "idle-rails", ...RAILS_BASE, cableUrl: TARGETS.railsSolidCable }, + baseline: {}, + driftThresholdPct: 60, + }, + { + id: "idle-rails-actioncable", + description: "Idle connections held, Rails + Action Cable (Redis)", + category: "idle", + endpoint: "bench-idle-anycable", + mode: "multi-shard", + numShards: 13, + perShardN: 4000, + params: { hold: 120, ramp: 200, stream: "idle-rails", ...RAILS_BASE, cableUrl: TARGETS.railsActionCable }, + baseline: {}, + driftThresholdPct: 60, + }, + { + id: "idle-rails-anycable", + description: "Idle connections held, Rails + AnyCable (Go gateway)", + category: "idle", + endpoint: "bench-idle-anycable", + mode: "multi-shard", + numShards: 13, + perShardN: 12000, + params: { hold: 120, ramp: 200, stream: "idle-rails", ...RAILS_EXT, cableUrl: TARGETS.railsAnyCable }, + baseline: {}, + driftThresholdPct: 60, + }, + { + id: "idle-rails-asynccable", + description: "Idle connections held, Rails + AsyncCable (Falcon)", + category: "idle", + endpoint: "bench-idle-anycable", + mode: "multi-shard", + numShards: 13, + perShardN: 4000, + params: { hold: 120, ramp: 200, stream: "idle-rails", ...RAILS_BASE, cableUrl: TARGETS.railsAsyncCable }, + baseline: {}, + driftThresholdPct: 60, + }, + + // Deploy survival. Redeploy the Rails service mid-test. Action Cable / Solid + // Cable run WebSockets in Puma, so a deploy drops every connection; AnyCable + // runs them in anycable-go, so redeploying the Rails RPC backend leaves the + // fleet connected (expected disconnected ~0). Native clients for the + // in-process adapters (clientLib=actioncable), matching the page's + // methodology. Baselines stay EMPTY on purpose: the published deploy + // numbers (recovery to 95% in ~13.5-13.8 s, ~96% reconnected) came from + // the SHARDED avalanche-multi drivers; these single-runner 5K specs are + // smoke checks, and pinning sharded-run numbers to them would compare + // different methodologies. Wiring sharded avalanche into the manifest is + // an open TODO. + { + id: "avalanche-rails-solidcable-5k", + description: "Avalanche: 5K Rails + Solid Cable clients, app redeploy", + category: "avalanche", + endpoint: "bench-avalanche-anycable", + mode: "avalanche", + redeployServiceName: "rails-solidcable", + params: { n: 5000, ramp: 200, prearm: 180, recoveryWait: 300, stream: "avalanche-rails-sc", ...RAILS_BASE, clientLib: "actioncable", cableUrl: TARGETS.railsSolidCable }, + baseline: {}, + driftThresholdPct: 100, + }, + { + id: "avalanche-rails-actioncable-5k", + description: "Avalanche: 5K Rails + Action Cable clients, app redeploy", + category: "avalanche", + endpoint: "bench-avalanche-anycable", + mode: "avalanche", + redeployServiceName: "rails-actioncable", + params: { n: 5000, ramp: 200, prearm: 180, recoveryWait: 300, stream: "avalanche-rails-ac", ...RAILS_BASE, clientLib: "actioncable", cableUrl: TARGETS.railsActionCable }, + baseline: {}, + driftThresholdPct: 100, + }, + { + id: "avalanche-rails-anycable-5k", + description: "Avalanche: 5K Rails + AnyCable clients, RPC backend redeploy (should survive)", + category: "avalanche", + endpoint: "bench-avalanche-anycable", + mode: "avalanche", + redeployServiceName: "rails-anycable", + params: { n: 5000, ramp: 200, prearm: 180, recoveryWait: 300, stream: "avalanche-rails-any", ...RAILS_EXT, cableUrl: TARGETS.railsAnyCable }, + baseline: {}, + driftThresholdPct: 100, + }, + { + id: "avalanche-rails-asynccable-5k", + description: "Avalanche: 5K Rails + AsyncCable (Falcon) clients, app redeploy", + category: "avalanche", + endpoint: "bench-avalanche-anycable", + mode: "avalanche", + redeployServiceName: "rails-asynccable", + params: { n: 5000, ramp: 200, prearm: 180, recoveryWait: 300, stream: "avalanche-rails-asc", ...RAILS_BASE, clientLib: "actioncable", cableUrl: TARGETS.railsAsyncCable }, + baseline: {}, + driftThresholdPct: 100, + }, ]; diff --git a/backend/src/bench/throughput-multi.ts b/backend/src/bench/throughput-multi.ts index e6f5849..46f2336 100644 --- a/backend/src/bench/throughput-multi.ts +++ b/backend/src/bench/throughput-multi.ts @@ -1,67 +1,39 @@ -// Multi-shard throughput benchmark — parallel to jitter-multi.ts. +// Multi-shard throughput benchmark. // -// The single-shard throughput test at 10K subscribers saturates the -// bench-runner Node.js event loop on @anycable/core, socket.io-client, -// or WS frame parsing, depending on protocol. That saturation inflates -// every protocol's measured p99 by a comparable amount — but unevenly -// across protocols (worst for @anycable/core), which distorts the -// cross-protocol comparison. +// Splits N subscribers across k shards; each shard publishes its own stream +// and receives its own fanout, keeping every runner under saturation while +// total server-side delivery work matches the single-shard test. // -// Fix: split N subscribers across k shards (k×N/k), each shard -// publishes its own stream and receives its own fanout. Per-shard -// runtime stays under saturation; total server-side delivery work -// matches the single-shard test (k × per-shard fanout = N total -// deliveries per broadcast cycle). -// -// Caveat: server-side per-broadcast overhead (e.g. AnyCable's broker -// write) happens k times more often than in single-shard, because each -// shard's publisher writes its own stream. For the page, treat the -// numbers here as "per-protocol fanout cost at 2500 subs, parallelized -// across k streams". They're closer to truth than single-shard 10K but -// not identical to "one fanout to 10K subs". See the footnote in the -// throughput table. +// Caveat: server-side per-broadcast overhead (e.g. AnyCable's broker write) +// happens k times more often than in single-shard, because each shard's +// publisher writes its own stream. Treat the numbers as "per-protocol +// fanout cost at N/k subs, parallelized across k streams" (see the +// footnote in the throughput table). Rate-matched: per-shard messages stay +// TOTAL_MESSAGES so the aggregate publish rate scales with k by design — +// state the shard count next to any published number. // // Usage: -// SHARDS=https://br-1.up.railway.app,https://br-2.up.railway.app,... \ +// SHARDS=https://br-1.up.railway.app,... \ // TOTAL=10000 RAMP_PER_SEC=200 \ // PROTOCOL=anycable \ # or socketio | socketio-csr | uws -// TOTAL_MESSAGES=100 INTERVAL_MS=10 \ # default 100 msgs × 10K subs = 1M +// TOTAL_MESSAGES=100 INTERVAL_MS=10 \ // PUBLISHER=pool PUBLISHER_CONCURRENCY=16 \ // DRAIN_SEC=30 SAMPLES_CAP=5000 \ -// CABLE_URL=... BROADCAST_URL=... SERVER_URL=... UWS_WS_URL=... UWS_HTTP_URL=... \ // tsx src/bench/throughput-multi.ts +// +// Target overrides come from lib/core/multi-shard.ts (CABLE_URL, +// BROADCAST_URL, CHANNEL, AC_PROTOCOL, SERVER_URL, UWS_WS_URL, UWS_HTTP_URL). -import { writeFileSync } from "node:fs"; -import { Agent, setGlobalDispatcher } from "undici"; - -import { resultPath } from "../lib/core/results-dir.js"; -import { runShards, type ShardSpec } from "../lib/core/shard-coordinator.js"; import { - formatHumanReport, - mergeJitterResults, - type JitterResult, -} from "../lib/core/stats.js"; + ENDPOINTS, + parseProtocol, + parseShardUrls, + protocolTargetQuery, + runMultiShard, +} from "../lib/core/multi-shard.js"; -// Throughput runs are bounded by the slowest shard; bump fetch timeouts -// past Railway's proxy ceiling so the coordinator never times out before -// the work does. -setGlobalDispatcher( - new Agent({ headersTimeout: 30 * 60 * 1000, bodyTimeout: 30 * 60 * 1000 }), -); - -const shardsCsv = process.env.SHARDS; -if (!shardsCsv) { - console.error("SHARDS env var required (comma-separated bench-runner base URLs)"); - process.exit(1); -} -const shardUrls = shardsCsv - .split(",") - .map((s) => s.trim()) - .filter(Boolean); -if (shardUrls.length === 0) { - console.error("SHARDS must contain at least one URL"); - process.exit(1); -} +const shardUrls = parseShardUrls(); +const protocol = parseProtocol(); const totalClients = parseInt(process.env.TOTAL || "10000", 10); const perShardN = Math.ceil(totalClients / shardUrls.length); @@ -75,166 +47,50 @@ const publisherConcurrency = parseInt( process.env.PUBLISHER_CONCURRENCY || "16", 10, ); -const protocol = (process.env.PROTOCOL || "anycable").toLowerCase(); - -const PROTOCOL_TO_ENDPOINT: Record = { - anycable: "bench-throughput-anycable", - socketio: "bench-throughput-socketio", - "socketio-csr": "bench-throughput-socketio-csr", - uws: "bench-throughput-uws", -}; -const endpoint = PROTOCOL_TO_ENDPOINT[protocol]; -if (!endpoint) { - console.error( - `PROTOCOL must be one of: ${Object.keys(PROTOCOL_TO_ENDPOINT).join(", ")} (got "${protocol}")`, - ); - process.exit(1); -} -// Per-protocol URL overrides, forwarded to each shard. Mirrors the -// surface that jitter-multi.ts exposes. -const protocolQuery: Record = {}; -if (protocol === "anycable") { - if (process.env.CABLE_URL) protocolQuery.cableUrl = process.env.CABLE_URL; - if (process.env.BROADCAST_URL) - protocolQuery.broadcastUrl = process.env.BROADCAST_URL; -} -if (protocol === "socketio" || protocol === "socketio-csr") { - if (process.env.SERVER_URL) protocolQuery.serverUrl = process.env.SERVER_URL; -} -if (protocol === "uws") { - if (process.env.UWS_WS_URL) protocolQuery.wsUrl = process.env.UWS_WS_URL; - if (process.env.UWS_HTTP_URL) protocolQuery.httpUrl = process.env.UWS_HTTP_URL; -} +const targetQuery = protocolTargetQuery(protocol); console.log( `Multi-shard throughput: ${shardUrls.length} shards × ${perShardN} = ${shardUrls.length * perShardN} clients (target: ${totalClients})`, ); console.log( - `Protocol: ${protocol} Endpoint: /${endpoint} Stream prefix: tp-multi-${Date.now()}`, -); -console.log( - `Per-shard: total=${totalMessages} intervalMs=${intervalMs} ramp=${rampPerSec}/s drain=${drainSec}s`, -); -console.log( - `Publisher: ${publisher} (concurrency ${publisherConcurrency}) Samples: ${samplesCap} per shard`, + `Protocol: ${protocol} Per-shard: total=${totalMessages} intervalMs=${intervalMs} ramp=${rampPerSec}/s drain=${drainSec}s Publisher: ${publisher}×${publisherConcurrency}`, ); console.log(""); -// Each shard owns its own stream so its publisher's broadcasts fan out -// only to its own subs. Shared streams would inflate every shard's -// deliveryRate denominator with other shards' work. -const runStamp = Date.now(); -const shardSpecs: ShardSpec[] = shardUrls.map((url, i) => ({ - url, - label: `shard-${i + 1}`, - endpoint, - query: { +const run = await runMultiShard({ + testType: "throughput", + protocol, + endpoint: ENDPOINTS.throughput[protocol], + shardUrls, + // Runner reads `intervalMs` (throughputParamsFromQuery). If this key ever + // drifts, the params echo in the enqueue response fails the shard before + // the run starts — that is the guard this exact line once needed. + shardQuery: (i, runStamp) => ({ n: perShardN, total: totalMessages, - interval: intervalMs, + intervalMs, ramp: rampPerSec, drain: drainSec, publisher, publisherConcurrency, samplesCap, stream: `tp-multi-${runStamp}-s${i + 1}`, - ...protocolQuery, + ...targetQuery, + }), + validity: { perShardN }, + meta: { + totalClients, + perShardN, + totalMessages, + intervalMs, + rampPerSec, + drainSec, + publisher, + publisherConcurrency, + samplesCap, + targetQuery, }, -})); - -const startedAt = new Date(); -const outcomes = await runShards(shardSpecs, { - pollIntervalMs: 5000, - pollLogLines: 30, - printProgress: true, }); -const endedAt = new Date(); - -const successes = outcomes.filter( - (o): o is typeof o & { result: JitterResult } => - o.status === "done" && o.result !== undefined, -); -const failures = outcomes.filter((o) => o.status !== "done"); - -console.log(""); -console.log(`=== Per-shard === (${successes.length} succeeded / ${outcomes.length})`); -for (const o of outcomes) { - if (o.status === "done" && o.result) { - const r = o.result; - console.log( - ` ${o.spec.label}: clients=${r.clients} delivery=${r.deliveryRatePct}% p50=${r.latencyOverMinMs.p50}ms p99=${r.latencyOverMinMs.p99}ms (${(o.durationMs / 1000).toFixed(1)}s)`, - ); - } else { - console.log(` ${o.spec.label}: FAILED — ${o.error || "unknown"}`); - } -} - -if (failures.length > 0) { - console.log(""); - console.log( - `!! ${failures.length} shard(s) failed; reported merge covers the remaining ${successes.length}.`, - ); -} - -if (successes.length === 0) { - console.error("All shards failed; no merged result to report."); - process.exit(1); -} - -// stats.ts:mergeJitterResults requires latencySamplesSorted on every -// shard. Sync-mode shards omit it. We pass samplesCap above so async -// shards include it, but log a friendly hint if a sync shard slipped -// through. -try { - const merged = mergeJitterResults( - `${protocol}-multi-${shardUrls.length}x${perShardN}`, - successes.map((o) => o.result), - ); - console.log(formatHumanReport(`Merged (${shardUrls.length} shards)`, merged)); - const outPath = resultPath( - `throughput-multi-${protocol}-${shardUrls.length}x${perShardN}-${startedAt.toISOString().replace(/[:.]/g, "-")}.json`, - ); - writeFileSync( - outPath, - JSON.stringify( - { - protocol, - shards: shardUrls.length, - perShardN, - totalClients: shardUrls.length * perShardN, - startedAt: startedAt.toISOString(), - endedAt: endedAt.toISOString(), - params: { - totalMessages, - intervalMs, - rampPerSec, - drainSec, - publisher, - publisherConcurrency, - samplesCap, - }, - perShard: outcomes.map((o) => ({ - label: o.spec.label, - url: o.spec.url, - status: o.status, - jobId: o.jobId, - durationMs: o.durationMs, - result: o.result, - error: o.error, - })), - merged, - }, - null, - 2, - ), - ); - console.log(`\nWrote merged JSON: ${outPath}`); - process.exit(merged.lostDeliveries > 0 ? 1 : 0); -} catch (e) { - console.error( - `\nMerge failed (${(e as Error).message}); per-shard summaries above are still valid.`, - ); - process.exit(0); -} +process.exit(run.exitCode); diff --git a/backend/src/lib/avalanche-anycable-runner.ts b/backend/src/lib/avalanche-anycable-runner.ts new file mode 100644 index 0000000..ab3412e --- /dev/null +++ b/backend/src/lib/avalanche-anycable-runner.ts @@ -0,0 +1,216 @@ +// Avalanche runner for the Action Cable protocol (AnyCable, Action Cable, +// Solid Cable). Connects N @anycable/core cables, waits for an externally +// triggered redeploy of the target service, and measures how the fleet +// recovers. Mirrors runAvalancheSocketio so results land in the same +// AvalancheResult shape and the rebaseline classifier works unchanged. +// +// The architectural contrast this captures: +// - Action Cable / Solid Cable terminate WebSockets in the Puma process, +// so redeploying the Rails app drops every connection (disconnected ~= N) +// and they must reconnect. +// - AnyCable terminates WebSockets in the anycable-go gateway; redeploying +// the Rails RPC backend leaves the gateway (and the held connections) +// untouched, so disconnected stays ~0 — the fleet survives the deploy. +// +// Listeners are attached at cable creation, before any redeploy can fire, so +// the disconnect that signals the restart is never missed. + +import WebSocket from "ws"; +import { createCable } from "@anycable/core"; + +import { percentile } from "./core/stats.js"; +import { settleAfterRamp } from "./core/timing.js"; +import { ActionCable } from "./core/actioncable-node.js"; +import type { AvalancheParams, AvalancheResult } from "./avalanche-runner.js"; + +export interface AvalancheAnycableUrls { + cableUrl: string; + channel?: string; + acProtocol?: string; + // "actioncable" drives the official @rails/actioncable client (native + // reconnect, base protocol) for Action Cable / Solid Cable / Async::Cable; + // default @anycable/core for AnyCable. + clientLib?: "anycable" | "actioncable"; +} + +export async function runAvalancheAnycable( + p: AvalancheParams, + urls: AvalancheAnycableUrls +): Promise { + const protocol = urls.acProtocol ?? "actioncable-v1-ext-json"; + const channelName = urls.channel ?? "$pubsub"; + console.log( + `[avalanche-ac] target=${urls.cableUrl} channel=${channelName} proto=${protocol} n=${p.n} ramp=${p.rampPerSec}/s prearm=${p.prearmSec}s recoveryWait=${p.recoveryWaitSec}s` + ); + + const conns: { disconnect(): void }[] = []; + const useActionCable = urls.clientLib === "actioncable"; + const startedAt = Date.now(); + + let initiallyConnected = 0; + let initialConnectDone = false; + let tearingDown = false; + + let disconnected = 0; + let firstDisconnectAt = 0; + let allDisconnectedAt = 0; + let restartDetectedAt = 0; + + let reconnected = 0; + let firstReconnectAt = 0; + let allReconnectedAt = 0; + const reconnectTimes: number[] = []; + + // Per-connection up/down state so accounting is idempotent: @anycable/core + // fires both "disconnect" and "close" on a single drop, and clients can emit + // repeat events, so we only count actual transitions (up->down, down->up) + // rather than raw events. A first connect during the ramp counts an initial + // connection; a connect after a detected restart counts a recovery. + const up: boolean[] = new Array(p.n).fill(false); + const handleUp = (i: number) => { + if (tearingDown) return; + if (up[i]) return; // already up — ignore duplicate connect events + up[i] = true; + if (!initialConnectDone) { + initiallyConnected++; + return; + } + if (restartDetectedAt > 0) { + reconnected++; + const now = Date.now(); + reconnectTimes.push(now - restartDetectedAt); + if (reconnected === 1) firstReconnectAt = now; + if (reconnected >= initiallyConnected * 0.95 && !allReconnectedAt) { + allReconnectedAt = now; + } + } + }; + const handleDown = (i: number) => { + if (tearingDown || !initialConnectDone) return; + if (!up[i]) return; // already down — collapse close+disconnect into one drop + up[i] = false; + disconnected++; + const now = Date.now(); + if (disconnected === 1) { + firstDisconnectAt = now; + restartDetectedAt = now; + } + if (disconnected === initiallyConnected) allDisconnectedAt = now; + }; + + for (let i = 0; i < p.n; i++) { + if (useActionCable) { + // Official Rails client — recovers on its own native monitor after the + // deploy drops it (no forced reconnect), so we measure its real recovery. + const consumer = ActionCable.createConsumer(urls.cableUrl); + consumer.subscriptions.create( + { channel: channelName, stream_name: p.stream }, + { + connected() { + handleUp(i); + }, + disconnected() { + handleDown(i); + }, + } + ); + conns.push({ disconnect: () => consumer.disconnect() }); + } else { + const cable = createCable(urls.cableUrl, { + websocketImplementation: WebSocket as unknown as typeof globalThis.WebSocket, + protocol: protocol as never, + logLevel: "error" as never, + }); + // Both "disconnect" and "close" can fire for a single drop; handleDown is + // idempotent per connection, so the drop is counted once. + cable.on("connect", () => handleUp(i)); + cable.on("disconnect", () => handleDown(i)); + cable.on("close", () => handleDown(i)); + // Subscribing triggers the connection. "$pubsub" -> streamFrom (signed + // pub/sub), any other channel -> a real Rails channel with { stream_name }. + if (channelName !== "$pubsub") { + cable.subscribeTo(channelName, { stream_name: p.stream }); + } else { + cable.streamFrom(p.stream); + } + conns.push({ disconnect: () => cable.disconnect() }); + } + + if ((i + 1) % p.rampPerSec === 0) { + await new Promise((r) => setTimeout(r, 1000)); + if ((i + 1) % 1000 === 0) console.log(`[avalanche-ac] ramped ${i + 1}/${p.n}`); + } + } + + await settleAfterRamp(); + initialConnectDone = true; + const rampElapsedMs = Date.now() - startedAt; + console.log( + `[avalanche-ac] all ramped (${rampElapsedMs}ms): ${initiallyConnected}/${p.n} connected` + ); + console.log(`[avalanche-ac] ready for redeploy — caller has ${p.prearmSec}s + recovery`); + + const armDeadline = Date.now() + p.prearmSec * 1000; + while (Date.now() < armDeadline && restartDetectedAt === 0) { + await new Promise((r) => setTimeout(r, 200)); + } + + if (restartDetectedAt === 0) { + // No disconnect observed during the window. For AnyCable this is the + // expected, healthy outcome: the gateway held every connection across the + // Rails redeploy. + console.log( + `[avalanche-ac] no disconnect within ${p.prearmSec}s — connections survived the deploy` + ); + } else { + console.log( + `[avalanche-ac] disconnect detected — waiting for reconnects (up to ${p.recoveryWaitSec}s)` + ); + const recoveryDeadline = restartDetectedAt + p.recoveryWaitSec * 1000; + while (Date.now() < recoveryDeadline) { + if (reconnected >= initiallyConnected * 0.95) break; + await new Promise((r) => setTimeout(r, 500)); + } + if (!allReconnectedAt) allReconnectedAt = Date.now(); + } + + tearingDown = true; + for (const c of conns) { + try { + c.disconnect(); + } catch { + /* ignore */ + } + } + + reconnectTimes.sort((a, b) => a - b); + const recoveryTimeMs = restartDetectedAt > 0 ? allReconnectedAt - restartDetectedAt : 0; + const neverReconnected = Math.max(0, initiallyConnected - reconnected); + + return { + clients: p.n, + initiallyConnected, + disconnected, + reconnected, + reconnectRatePct: + initiallyConnected > 0 + ? Number(((reconnected / initiallyConnected) * 100).toFixed(2)) + : 0, + neverReconnected, + neverReconnectedPct: + initiallyConnected > 0 + ? Number(((neverReconnected / initiallyConnected) * 100).toFixed(2)) + : 0, + disconnectSpreadMs: allDisconnectedAt > 0 ? allDisconnectedAt - firstDisconnectAt : 0, + recoveryTimeMs, + reconnectMs: { + p50: percentile(reconnectTimes, 50), + p95: percentile(reconnectTimes, 95), + p99: percentile(reconnectTimes, 99), + max: percentile(reconnectTimes, 100), + }, + totalDowntimeMs: recoveryTimeMs, + rampElapsedMs, + totalElapsedMs: Date.now() - startedAt, + }; +} diff --git a/backend/src/lib/core/actioncable-node.ts b/backend/src/lib/core/actioncable-node.ts new file mode 100644 index 0000000..e45df48 --- /dev/null +++ b/backend/src/lib/core/actioncable-node.ts @@ -0,0 +1,24 @@ +// Run the official @rails/actioncable client under Node: inject a WebSocket +// implementation and stub the browser globals its ConnectionMonitor touches +// (addEventListener/removeEventListener for online/offline + visibility events, +// document.visibilityState). Import this module before creating any consumer. +import WebSocket from "ws"; +import * as ActionCable from "@rails/actioncable"; + +{ + const g = globalThis as unknown as Record; + if (typeof g.addEventListener !== "function") g.addEventListener = () => {}; + if (typeof g.removeEventListener !== "function") + g.removeEventListener = () => {}; + if (typeof g.document === "undefined") { + g.document = { + visibilityState: "visible", + addEventListener: () => {}, + removeEventListener: () => {}, + }; + } +} +(ActionCable.adapters as { WebSocket: unknown }).WebSocket = + WebSocket as unknown; + +export { ActionCable }; diff --git a/backend/src/lib/core/multi-shard.ts b/backend/src/lib/core/multi-shard.ts new file mode 100644 index 0000000..5f37b73 --- /dev/null +++ b/backend/src/lib/core/multi-shard.ts @@ -0,0 +1,365 @@ +// The one multi-shard primitive. +// +// Every multi-shard driver (jitter, throughput, idle, whispers, avalanche) +// used to hand-roll the same four things: SHARDS parsing, the protocol → +// target-override query mapping, the fan-out/poll loop, and result merging. +// The mapping was the dangerous one — the channel/acProtocol passthrough +// gap was rediscovered in three drivers, and a param-name mismatch once ran +// a whole throughput suite at the default rate. This module is the single +// place all of that lives; drivers reduce to "pick endpoint, pick per-shard +// params, pick merge". +// +// It also runs the validity checks (lib/core/validity.ts) on every result +// and writes them into the result JSON, so a run that measured the rig +// instead of the server flags itself. + +import { writeFileSync } from "node:fs"; +import { Agent, setGlobalDispatcher } from "undici"; + +import { benchRunnerFetch } from "./bench-runner-client.js"; +import { resultPath } from "./results-dir.js"; +import { runShards, type ShardOutcome, type ShardSpec } from "./shard-coordinator.js"; +import { + formatHumanReport, + mergeJitterResults, + type JitterResult, +} from "./stats.js"; +import { + formatValidityReport, + hasFatal, + validateResult, + validateShardSet, + type ValidityContext, + type ValidityFlag, +} from "./validity.js"; + +// Enqueue/poll fetches are short, but old runners that ignore ?async=1 +// block the POST for the whole test. Pad timeouts so mixed fleets work. +setGlobalDispatcher( + new Agent({ headersTimeout: 30 * 60 * 1000, bodyTimeout: 30 * 60 * 1000 }), +); + +// --------------------------------------------------------------------------- +// Shared env parsing + +export function parseShardUrls(): string[] { + const csv = process.env.SHARDS; + if (!csv) { + console.error( + "SHARDS env var required (comma-separated bench-runner base URLs)", + ); + process.exit(1); + } + const urls = csv + .split(",") + .map((s) => s.trim()) + .filter(Boolean); + if (urls.length === 0) { + console.error("SHARDS must contain at least one URL"); + process.exit(1); + } + return urls; +} + +export type Protocol = "anycable" | "socketio" | "socketio-csr" | "uws"; + +export function parseProtocol(): Protocol { + const p = (process.env.PROTOCOL || "anycable").toLowerCase(); + if (p !== "anycable" && p !== "socketio" && p !== "socketio-csr" && p !== "uws") { + console.error( + `PROTOCOL must be one of: anycable, socketio, socketio-csr, uws (got "${p}")`, + ); + process.exit(1); + } + return p; +} + +// THE protocol → target-override mapping. Reads the same env vars as the +// single-shard drivers and forwards them as query params. Add a new +// override here once and every multi-shard driver gets it — this exact +// list was previously re-implemented (incompletely) per driver. +export function protocolTargetQuery(protocol: Protocol): Record { + const q: Record = {}; + if (protocol === "anycable") { + if (process.env.CABLE_URL) q.cableUrl = process.env.CABLE_URL; + if (process.env.BROADCAST_URL) q.broadcastUrl = process.env.BROADCAST_URL; + // Rails targets subscribe to a real channel over the base or extended + // Action Cable wire protocol; nodejs $pubsub targets leave these unset. + if (process.env.CHANNEL) q.channel = process.env.CHANNEL; + if (process.env.AC_PROTOCOL) q.acProtocol = process.env.AC_PROTOCOL; + // Reconnect matrix: default = each client's stock backoff (real UX), + // tuned = the uniform aggressive profile (server resume ceiling). + if (process.env.RECONNECT_MODE) q.reconnectMode = process.env.RECONNECT_MODE; + if (process.env.RECONNECT_BASE_MS) + q.reconnectBaseMs = process.env.RECONNECT_BASE_MS; + // CLIENT_LIB=actioncable drives @rails/actioncable instead of + // @anycable/core (Action Cable / Solid Cable / Async::Cable targets). + if (process.env.CLIENT_LIB) q.clientLib = process.env.CLIENT_LIB; + if (process.env.NATS_URL) q.natsUrl = process.env.NATS_URL; + if (process.env.NATS_SUBJECT) q.natsSubject = process.env.NATS_SUBJECT; + } + if (protocol === "socketio" || protocol === "socketio-csr") { + if (process.env.SERVER_URL) q.serverUrl = process.env.SERVER_URL; + } + if (protocol === "uws") { + if (process.env.UWS_WS_URL) q.wsUrl = process.env.UWS_WS_URL; + if (process.env.UWS_HTTP_URL) q.httpUrl = process.env.UWS_HTTP_URL; + } + return q; +} + +export const ENDPOINTS: Record> = { + jitter: { + anycable: "bench-jitter-anycable", + socketio: "bench-jitter-socketio", + "socketio-csr": "bench-jitter-socketio-csr", + uws: "bench-jitter-uws", + }, + throughput: { + anycable: "bench-throughput-anycable", + socketio: "bench-throughput-socketio", + "socketio-csr": "bench-throughput-socketio-csr", + uws: "bench-throughput-uws", + }, + whispers: { + anycable: "bench-whispers-anycable", + socketio: "bench-whispers-socketio", + "socketio-csr": "bench-whispers-socketio", + uws: "bench-whispers-uws", + }, + idle: { + anycable: "bench-idle-anycable", + socketio: "bench-idle-socketio", + "socketio-csr": "bench-idle-socketio", + uws: "bench-idle-uws", + }, + avalanche: { + anycable: "bench-avalanche-anycable", + socketio: "bench-avalanche-socketio", + "socketio-csr": "bench-avalanche-socketio", + uws: "bench-avalanche-uws", + }, +}; + +// --------------------------------------------------------------------------- +// Health precheck + +export interface ShardHealth { + url: string; + ok: boolean; + detail: string; +} + +// GET /health on every shard before enqueueing anything. Catches dead +// shards, missing domains, and (via a follow-up authenticated probe) +// stale tokens and pre-async images — before the run, not during. +export async function checkShardHealth(urls: string[]): Promise { + return Promise.all( + urls.map(async (url): Promise => { + try { + const res = await fetch(`${url}/health`, { + signal: AbortSignal.timeout(10_000), + }); + if (!res.ok) return { url, ok: false, detail: `health HTTP ${res.status}` }; + const body = (await res.json()) as { authRequired?: boolean }; + // Authenticated probe: /jobs/nope should be a JSON 404 with a valid + // token (401 = bad/missing token; HTML 404 = pre-async old image). + const probe = await benchRunnerFetch(`${url}/jobs/nope`, { + signal: AbortSignal.timeout(10_000), + }); + if (probe.status === 401) { + return { url, ok: false, detail: "token rejected (401) — BENCH_RUNNER_TOKEN mismatch" }; + } + if (probe.status !== 404) { + return { url, ok: false, detail: `job probe HTTP ${probe.status}` }; + } + // New images answer the probe with JSON ({"error":"job not found"}); + // pre-async images fall through to Express's HTML 404. Key on the + // body, not the content-type header (Railway's edge drops it on + // some HTTP/1.1 responses). + const probeBody = await probe.text(); + try { + JSON.parse(probeBody); + } catch { + return { url, ok: false, detail: "job probe returned non-JSON 404 — old image without the async job API" }; + } + return { + url, + ok: true, + detail: body.authRequired ? "ok (auth on)" : "ok (AUTH OFF)", + }; + } catch (err) { + return { + url, + ok: false, + detail: err instanceof Error ? err.message : String(err), + }; + } + }), + ); +} + +// --------------------------------------------------------------------------- +// The primitive + +export interface MultiShardConfig { + // Used in labels and the result filename: "jitter" | "throughput" | ... + testType: string; + protocol: Protocol; + endpoint: string; + shardUrls: string[]; + // Per-shard query params. `i` is 0-based shard index; `runStamp` is one + // timestamp shared by the whole run (use it to build unique streams). + shardQuery: (i: number, runStamp: number) => Record; + // Merge per-shard JitterResults. Defaults to mergeJitterResults. + merge?: (label: string, results: JitterResult[]) => JitterResult; + // Context for the validity checks. + validity?: ValidityContext; + // Extra fields recorded into the result JSON (test params, notes). + meta?: Record; + // Skip the pre-run health sweep (already done externally). + skipHealthCheck?: boolean; + shardTimeoutMs?: number; +} + +export interface MultiShardRun { + outcomes: ShardOutcome[]; + successes: JitterResult[]; + merged: JitterResult | null; + flags: ValidityFlag[]; + outPath: string | null; + // Suggested process exit code: 0 clean, 1 lost deliveries or failed + // shards, 2 fatal validity flags (do not publish). + exitCode: number; +} + +export async function runMultiShard(cfg: MultiShardConfig): Promise { + const k = cfg.shardUrls.length; + const runStamp = Date.now(); + const label = `${cfg.protocol}-${cfg.testType}-multi-${k}`; + + if (!cfg.skipHealthCheck) { + console.log(`Health sweep: ${k} shard(s)...`); + const health = await checkShardHealth(cfg.shardUrls); + const bad = health.filter((h) => !h.ok); + for (const h of bad) console.error(` ✗ ${h.url}: ${h.detail}`); + if (bad.length > 0) { + console.error( + `${bad.length}/${k} shard(s) unhealthy. Fix or drop them from SHARDS before burning a run.`, + ); + process.exit(1); + } + console.log(` ✓ all ${k} shards healthy`); + } + + const specs: ShardSpec[] = cfg.shardUrls.map((url, i) => ({ + url, + label: `shard-${i + 1}`, + endpoint: cfg.endpoint, + query: cfg.shardQuery(i, runStamp), + })); + + const startedAt = new Date(); + const outcomes = await runShards(specs, { + pollIntervalMs: 5000, + pollLogLines: 30, + printProgress: true, + shardTimeoutMs: cfg.shardTimeoutMs, + }); + const endedAt = new Date(); + + const successOutcomes = outcomes.filter( + (o): o is typeof o & { result: JitterResult } => + o.status === "done" && o.result !== undefined, + ); + const successes = successOutcomes.map((o) => o.result); + const failures = outcomes.filter((o) => o.status !== "done"); + + console.log(""); + console.log(`=== Per-shard === (${successes.length} succeeded / ${outcomes.length})`); + for (const o of outcomes) { + if (o.status === "done" && o.result) { + const r = o.result; + console.log( + ` ${o.spec.label}: clients=${r.clients} delivery=${r.deliveryRatePct}% p50=${r.latencyOverMinMs.p50}ms p99=${r.latencyOverMinMs.p99}ms (${(o.durationMs / 1000).toFixed(1)}s)`, + ); + } else { + console.log(` ${o.spec.label}: FAILED — ${o.error || "unknown"}`); + } + } + if (failures.length > 0) { + console.log( + `\n!! ${failures.length} shard(s) failed; the merge covers the remaining ${successes.length}. A partial fleet is a different test — rerun unless this was expected.`, + ); + } + if (successes.length === 0) { + console.error("All shards failed; nothing to merge."); + return { + outcomes, + successes, + merged: null, + flags: [], + outPath: null, + exitCode: 1, + }; + } + + // Merge. Sync-mode shards lack latencySamplesSorted and make + // mergeJitterResults throw; per-shard results above stay valid. + let merged: JitterResult | null = null; + try { + const mergeFn = cfg.merge ?? mergeJitterResults; + merged = mergeFn(label, successes); + console.log(formatHumanReport(`Merged (${successes.length} shards)`, merged)); + } catch (e) { + console.error( + `\nMerge failed (${(e as Error).message}); per-shard summaries above are still valid.`, + ); + } + + // Validity: per-shard set checks + merged-result checks. + const flags: ValidityFlag[] = [ + ...validateShardSet(successes, cfg.validity), + ...(merged ? validateResult(merged, cfg.validity) : []), + ]; + console.log(formatValidityReport(flags)); + + const outPath = resultPath( + `${cfg.testType}-multi-${cfg.protocol}-${k}shards-${startedAt.toISOString().replace(/[:.]/g, "-")}.json`, + ); + writeFileSync( + outPath, + JSON.stringify( + { + testType: cfg.testType, + protocol: cfg.protocol, + endpoint: cfg.endpoint, + shards: k, + startedAt: startedAt.toISOString(), + endedAt: endedAt.toISOString(), + meta: cfg.meta ?? {}, + validity: flags, + perShard: outcomes.map((o) => ({ + label: o.spec.label, + url: o.spec.url, + status: o.status, + jobId: o.jobId, + durationMs: o.durationMs, + result: o.result, + error: o.error, + })), + merged, + }, + null, + 2, + ), + ); + console.log(`\nWrote result JSON: ${outPath}`); + + const exitCode = hasFatal(flags) + ? 2 + : failures.length > 0 || (merged?.lostDeliveries ?? 0) > 0 + ? 1 + : 0; + return { outcomes, successes, merged, flags, outPath, exitCode }; +} diff --git a/backend/src/lib/core/railway-api.ts b/backend/src/lib/core/railway-api.ts index f8cb1db..3fd613c 100644 --- a/backend/src/lib/core/railway-api.ts +++ b/backend/src/lib/core/railway-api.ts @@ -59,7 +59,7 @@ export async function fetchMetric(args: FetchMetricArgs): Promise { start: args.startDate, end: args.endDate, measurement: args.measurement, - sampleRate: args.sampleRate ?? 30, + sampleRate: args.sampleRate ?? 60, }, }), }); diff --git a/backend/src/lib/core/railway-fleet.ts b/backend/src/lib/core/railway-fleet.ts new file mode 100644 index 0000000..6a6a596 --- /dev/null +++ b/backend/src/lib/core/railway-fleet.ts @@ -0,0 +1,217 @@ +// Railway GraphQL helpers for fleet inspection (preflight / fleet diff / +// watchdog). Read-only: nothing here mutates services. +// +// Two operational rules learned the hard way, encoded here: +// - never swallow GraphQL errors (an auth expiry once masqueraded as +// rate limiting for an hour because stderr went to /dev/null) — every +// helper throws with the raw error payload; +// - Railway's schema drifts (the CLI's own `scale` subcommand panics on +// it), so callers should catch and report, not assume. + +import { createHash } from "node:crypto"; +import { readFileSync } from "node:fs"; + +import { readRailwayToken } from "./railway-api.js"; + +const ENDPOINT = "https://backboard.railway.com/graphql/v2"; + +export async function gql( + query: string, + variables: Record, + token: string, +): Promise { + const res = await fetch(ENDPOINT, { + method: "POST", + headers: { + Authorization: `Bearer ${token}`, + "Content-Type": "application/json", + // Railway's edge rejects unfamiliar user agents (Python's default UA + // gets a Cloudflare 403); curl's is known-good. + "User-Agent": "curl/8.4.0", + // Ask for uncompressed responses: Railway's edge otherwise sends + // gzip that this fetch path does not always decompress (the + // railway-metrics helper crashed on exactly this). + "Accept-Encoding": "identity", + }, + body: JSON.stringify({ query, variables }), + }); + const text = await res.text(); + let json: { data?: T; errors?: unknown }; + try { + json = JSON.parse(text); + } catch { + throw new Error( + `Railway GraphQL returned non-JSON (HTTP ${res.status}): ${text.slice(0, 300)}`, + ); + } + if (json.errors) { + throw new Error( + `Railway GraphQL error: ${JSON.stringify(json.errors).slice(0, 500)}\n` + + `(auth expired? run 'railway login'; schema drift? print the raw error, do not guess)`, + ); + } + if (!json.data) { + throw new Error(`Railway GraphQL returned no data (HTTP ${res.status})`); + } + return json.data; +} + +export interface FleetManifest { + project: string; + projectId?: string; + environment: string; + environmentId?: string; + runnerPattern: string; + excludedRunners: string[]; + sharedSecrets: string[]; + targets: Record< + string, + { + role: string; + expectEnv: string[]; + // Env values that make a run silently invalid (e.g. broadcast + // adapter nats): key → forbidden value. + forbidEnv?: Record; + notes: string; + } + >; +} + +export function readManifest(path: string): FleetManifest { + return JSON.parse(readFileSync(path, "utf-8")) as FleetManifest; +} + +export interface ResolvedProject { + projectId: string; + environmentId: string; +} + +// Resolve project + environment ids. Precedence: PROJECT_ID/ENVIRONMENT_ID +// env vars, then the ids pinned in fleet-manifest.json (the project lives +// in a team workspace, so the account-level `projects` listing does not +// see it), verified against the live project name. +export async function resolveProject( + manifest: FleetManifest, + token: string, +): Promise { + if (process.env.PROJECT_ID && process.env.ENVIRONMENT_ID) { + return { + projectId: process.env.PROJECT_ID, + environmentId: process.env.ENVIRONMENT_ID, + }; + } + if (!manifest.projectId || !manifest.environmentId) { + throw new Error( + "fleet-manifest.json needs projectId + environmentId (or set PROJECT_ID + ENVIRONMENT_ID). Find them: railway status --json | jq '{id, env: .environments.edges[0].node.id}'", + ); + } + const data = await gql<{ project: { id: string; name: string } }>( + `query P($id: String!) { project(id: $id) { id name } }`, + { id: manifest.projectId }, + token, + ); + if (data.project.name !== manifest.project) { + throw new Error( + `projectId ${manifest.projectId} resolves to "${data.project.name}", manifest says "${manifest.project}" — fix the manifest`, + ); + } + return { + projectId: manifest.projectId, + environmentId: manifest.environmentId, + }; +} + +export interface ServiceState { + serviceId: string; + serviceName: string; + deployment: { + id: string; + status: string; + createdAt: string; + } | null; +} + +export async function listServiceStates( + resolved: ResolvedProject, + token: string, +): Promise { + const data = await gql<{ + environment: { + serviceInstances: { + edges: Array<{ + node: { + serviceId: string; + serviceName: string; + latestDeployment: { + id: string; + status: string; + createdAt: string; + } | null; + }; + }>; + }; + }; + }>( + `query Env($id: String!) { + environment(id: $id) { + serviceInstances { + edges { + node { + serviceId + serviceName + latestDeployment { id status createdAt } + } + } + } + } + }`, + { id: resolved.environmentId }, + token, + ); + return data.environment.serviceInstances.edges.map((e) => ({ + serviceId: e.node.serviceId, + serviceName: e.node.serviceName, + deployment: e.node.latestDeployment, + })); +} + +// A deployment counts as live (billing) when its latest deployment is in a +// running-ish state. REMOVED = torn down; FAILED/CRASHED are not billing +// but are also not healthy. +const LIVE_STATUSES = new Set(["SUCCESS", "DEPLOYING", "BUILDING", "INITIALIZING", "WAITING"]); +const CHURN_STATUSES = new Set(["DEPLOYING", "BUILDING", "INITIALIZING", "WAITING"]); + +export function isLive(s: ServiceState): boolean { + return s.deployment !== null && LIVE_STATUSES.has(s.deployment.status); +} + +export function isChurning(s: ServiceState): boolean { + return s.deployment !== null && CHURN_STATUSES.has(s.deployment.status); +} + +// Decrypted service variables. Never print values — hash them (sha12) for +// cross-service comparison. +export async function getServiceVariables( + resolved: ResolvedProject, + serviceId: string, + token: string, +): Promise> { + const data = await gql<{ variables: Record }>( + `query Vars($projectId: String!, $environmentId: String!, $serviceId: String!) { + variables(projectId: $projectId, environmentId: $environmentId, serviceId: $serviceId) + }`, + { + projectId: resolved.projectId, + environmentId: resolved.environmentId, + serviceId, + }, + token, + ); + return data.variables ?? {}; +} + +export function sha12(value: string): string { + return createHash("sha256").update(value).digest("hex").slice(0, 12); +} + +export { readRailwayToken }; diff --git a/backend/src/lib/core/reconnect-strategies.ts b/backend/src/lib/core/reconnect-strategies.ts new file mode 100644 index 0000000..3609662 --- /dev/null +++ b/backend/src/lib/core/reconnect-strategies.ts @@ -0,0 +1,67 @@ +// Client reconnect-backoff strategies, one explicit function per real client +// library, plus a shared "tuned" profile. The jitter/recovery test runs in two +// modes so we can separate a client-library default from a server's actual +// resume/replay capability: +// +// DEFAULT — each server driven by its stock client's real reconnect backoff. +// This is what users actually experience out of the box. +// TUNED — every client uses one uniform aggressive-but-storm-safe backoff, +// so the recovery tail reflects the SERVER's resume speed, not the +// client's stock delay. +// +// The recovery-tail "p99" a jitter run reports is roughly (offline window) + +// (client reconnect delay) + (server replay). At scale the reconnect delay +// dominates, so which backoff the client ships is the single biggest lever. +// +// Per Vladimir's PR review: express these as explicit named functions rather +// than reparametrizing `backoffWithJitter` to fake a specific client (which +// gets cryptic). Each function is `(attempts) => delayMs`. + +import { backoffWithJitter } from "@anycable/core"; + +export type ReconnectMode = "default" | "tuned"; + +// Default base delay (ms) for the tuned profile's first reconnect attempt. +export const TUNED_BASE_MS = 500; + +// @rails/actioncable — a direct port of connection_monitor.js `getPollInterval` +// (staleThreshold = 6 s, reconnectionBackoffRate = 0.15). First reconnect +// floors around 6 s, the slowest stock reconnect in the comparison, which is +// why Action Cable's out-of-box recovery tail is the longest. +// Ref: rails/rails actioncable/app/javascript/action_cable/connection_monitor.js +export function actionCableStrategy(attempts: number): number { + const staleThreshold = 6; + const reconnectionBackoffRate = 0.15; + const backoff = Math.pow(1 + reconnectionBackoffRate, Math.min(attempts, 10)); + const jitterMax = attempts === 0 ? 1.0 : reconnectionBackoffRate; + const jitter = jitterMax * Math.random(); + return staleThreshold * 1000 * backoff * (1 + jitter); +} + +// @anycable/core — the library's built-in default is backoffWithJitter with the +// ping interval (3000 ms) as the base: first reconnect ~1.5–9 s (centered ~4.5 s). +// Exposed as a named strategy so the DEFAULT mode sets it explicitly. +export const anycableDefaultStrategy = backoffWithJitter(3000); + +// centrifuge-js — full-jitter, min 500 ms / max 20000 ms: first reconnect +// ~500–1000 ms, the most aggressive stock default. Centrifugo is not a target +// in this repo currently; kept for reference and reuse if it's re-added. +export function centrifugeStrategy(attempts: number): number { + const min = 500; + const max = 20000; + const cap = Math.min(max, min * Math.pow(2, Math.min(attempts, 31))); + return Math.min(max, min + Math.floor(Math.random() * cap)); +} + +// Shared TUNED / "best" profile, applied uniformly to every client. Aggressive +// first attempt (~baseMs) with x2 growth capped at 5 s and FULL jitter — fast +// enough to shrink the resume tail, but jittered + capped so a large fleet does +// not reconnect in lockstep (the reconnect storm the conservative stock +// defaults deliberately avoid; verify it holds at the test's scale). +export function makeTunedStrategy(baseMs: number = TUNED_BASE_MS): (attempts: number) => number { + const max = 5000; + return (attempts: number): number => { + const cap = Math.min(max, baseMs * Math.pow(2, Math.min(attempts, 10))); + return Math.min(max, baseMs + Math.random() * cap); + }; +} diff --git a/backend/src/lib/core/shard-coordinator.ts b/backend/src/lib/core/shard-coordinator.ts index 68e71d6..b455166 100644 --- a/backend/src/lib/core/shard-coordinator.ts +++ b/backend/src/lib/core/shard-coordinator.ts @@ -65,6 +65,13 @@ export interface RunShardsOptions { // Per-shard absolute ceiling. If a shard hasn't reported done by then, // we abandon polling on it and mark it failed. Default 60 min. shardTimeoutMs?: number; + // New-style runners echo {effectiveParams, unknownParams} in the 202 + // enqueue response. By default a shard fails fast when the runner + // reports it would ignore a param we sent, or when an echoed value + // differs from what we sent — both mean the test is about to run with + // different parameters than the driver believes. Set true only for a + // deliberate mixed-fleet run. + allowParamMismatch?: boolean; } interface ShardState { @@ -80,13 +87,35 @@ interface ShardState { lastPrintedLogIdx: number; } +// Compare what we sent against what the runner says it parsed. Only keys +// present under the same name on both sides are compared — parser-side +// names sometimes differ (msgs → totalMessages), and the unknown-params +// check covers keys the runner doesn't read at all. +function paramEchoMismatches( + sent: Record, + effective: Record, +): string[] { + const mismatches: string[] = []; + for (const [k, v] of Object.entries(sent)) { + if (!(k in effective)) continue; + const echoed = effective[k]; + if (echoed === undefined || echoed === null) continue; + if (String(echoed) !== String(v)) { + mismatches.push(`${k}: sent ${v}, runner parsed ${String(echoed)}`); + } + } + return mismatches; +} + // Enqueue returns one of three shapes: // - {jobId} new-style async bench-runner; caller polls /jobs/:id // - {syncResult} pre-async bench-runner ignored ?async=1 and returned // the full result inline. Treat as already-done. -// - {error} network / HTTP failure. +// - {error} network / HTTP failure, or a param the runner would +// ignore / misparse (fail fast before wasting the run). async function enqueueShard( shard: ShardSpec, + allowParamMismatch: boolean, ): Promise< { jobId: string } | { syncResult: unknown } | { error: string } > { @@ -103,6 +132,32 @@ async function enqueueShard( } const body = (await res.json()) as Record; if (typeof body.jobId === "string") { + // Params echo verification (new-style runners only). A shard that + // would silently ignore or default a param is failed here, before + // the fleet burns a full run measuring the wrong configuration. + const unknown = Array.isArray(body.unknownParams) + ? (body.unknownParams as string[]) + : undefined; + if (unknown === undefined) { + console.log( + ` ! ${shard.label}: runner does not echo params (old image?) — param verification skipped`, + ); + } else if (unknown.length > 0) { + const msg = `runner ignores query params: ${unknown.join(", ")} (endpoint /${shard.endpoint} does not read them; check the key names)`; + if (!allowParamMismatch) return { error: msg }; + console.log(` ! ${shard.label}: ${msg}`); + } + if (body.effectiveParams && typeof body.effectiveParams === "object") { + const mismatches = paramEchoMismatches( + shard.query, + body.effectiveParams as Record, + ); + if (mismatches.length > 0) { + const msg = `param echo mismatch: ${mismatches.join("; ")}`; + if (!allowParamMismatch) return { error: msg }; + console.log(` ! ${shard.label}: ${msg}`); + } + } return { jobId: body.jobId }; } // Old bench-runner: returned the full sync result instead of {jobId}. @@ -171,6 +226,7 @@ export async function runShards( const pollLogLines = opts.pollLogLines ?? 30; const printProgress = opts.printProgress ?? true; const shardTimeoutMs = opts.shardTimeoutMs ?? 60 * 60 * 1000; + const allowParamMismatch = opts.allowParamMismatch ?? false; console.log(`Enqueuing ${shards.length} shard(s)...`); const states: ShardState[] = shards.map((s) => ({ @@ -183,7 +239,7 @@ export async function runShards( await Promise.all( states.map(async (state) => { - const r = await enqueueShard(state.spec); + const r = await enqueueShard(state.spec, allowParamMismatch); if ("error" in r) { state.enqueueError = r.error; state.status = "failed"; diff --git a/backend/src/lib/core/validity.test.ts b/backend/src/lib/core/validity.test.ts new file mode 100644 index 0000000..82590ab --- /dev/null +++ b/backend/src/lib/core/validity.test.ts @@ -0,0 +1,115 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; + +import type { JitterResult } from "./stats.js"; +import { hasFatal, validateResult, validateShardSet } from "./validity.js"; + +function result(overrides: Partial = {}): JitterResult { + return { + label: "test", + elapsedMs: 100_000, + clients: 250, + connectedClients: 250, + publishedMessages: 120, + expectedDeliveries: 30_000, + receivedDeliveries: 30_000, + lostDeliveries: 0, + deliveryRatePct: 100, + deliveryRateOfConnectedPct: 100, + jitterEvents: 0, + avgJittersPerClient: 0, + csrResumes: 0, + csrResumeRatePct: null, + connectFailures: 0, + latencyRawMs: { avg: 10, p50: 8, p95: 20, p99: 30, max: 50 }, + latencyOverMinMs: { avg: 8, p50: 6, p95: 18, p99: 28, max: 48, skewFloor: 2 }, + latencySamples: 30_000, + runnerPeakRssMb: 200, + ...overrides, + }; +} + +test("clean result produces no flags", () => { + assert.deepEqual(validateResult(result()), []); +}); + +test("delivery over 100% is fatal (sender echo class)", () => { + const flags = validateResult(result({ deliveryRatePct: 111.11 })); + assert.ok(flags.some((f) => f.code === "delivery-over-100" && f.severity === "fatal")); +}); + +test("zero connected clients is fatal", () => { + const flags = validateResult(result({ connectedClients: 0 })); + assert.ok(flags.some((f) => f.code === "nothing-connected")); +}); + +test("zero published messages is fatal (silent 401 class)", () => { + const flags = validateResult(result({ publishedMessages: 0 })); + assert.ok(flags.some((f) => f.code === "nothing-published")); +}); + +test("negative skew floor is fatal (cross-clock class)", () => { + const flags = validateResult( + result({ + latencyOverMinMs: { avg: 8, p50: 6, p95: 18, p99: 28, max: 48, skewFloor: -3 }, + }), + ); + assert.ok(flags.some((f) => f.code === "negative-skew-floor")); +}); + +test("elapsed far beyond configured duration is fatal (event-loop saturation)", () => { + const flags = validateResult(result({ elapsedMs: 94_000 }), { + expectedDurationSec: 30, + }); + assert.ok(flags.some((f) => f.code === "elapsed-overrun")); +}); + +test("elapsed within 1.5x of configured duration passes", () => { + const flags = validateResult(result({ elapsedMs: 200_000 }), { + expectedDurationSec: 160, + }); + assert.ok(!flags.some((f) => f.code === "elapsed-overrun")); +}); + +test("connect failures above 1% warn", () => { + const flags = validateResult(result({ connectFailures: 10 })); + const flag = flags.find((f) => f.code === "connect-failures"); + assert.ok(flag); + assert.equal(flag.severity, "warn"); +}); + +test("received above expected is fatal (duplicate counting)", () => { + const flags = validateResult( + result({ receivedDeliveries: 33_000, deliveryRatePct: 110 }), + ); + assert.ok(flags.some((f) => f.code === "received-over-expected")); +}); + +test("uniform shard ceiling below requested N is fatal", () => { + const shards = [ + result({ connectedClients: 12_002, clients: 12_002 }), + result({ connectedClients: 12_002, clients: 12_002 }), + result({ connectedClients: 12_002, clients: 12_002 }), + ]; + const flags = validateShardSet(shards, { perShardN: 20_000 }); + assert.ok(flags.some((f) => f.code === "uniform-shard-ceiling")); + assert.ok(hasFatal(flags)); +}); + +test("uniform shard counts at requested N are fine", () => { + const shards = [ + result({ connectedClients: 250 }), + result({ connectedClients: 250 }), + ]; + assert.deepEqual(validateShardSet(shards, { perShardN: 250 }), []); +}); + +test("partial publisher silence across the fleet is fatal (secret drift)", () => { + const shards = [ + result(), + result({ publishedMessages: 0 }), + result({ publishedMessages: 0 }), + ]; + const flags = validateShardSet(shards, { perShardN: 250 }); + assert.ok(flags.some((f) => f.code === "partial-publisher-silence")); +}); diff --git a/backend/src/lib/core/validity.ts b/backend/src/lib/core/validity.ts new file mode 100644 index 0000000..36aec2a --- /dev/null +++ b/backend/src/lib/core/validity.ts @@ -0,0 +1,171 @@ +// Result self-flagging for multi-shard runs. +// +// Every retracted number in this project's history matched one of a small +// set of signatures: a saturated load generator, a dead or misconfigured +// target, a silently-401'd publisher, cross-clock skew, or a semantic bug +// (echo, duplicate counting). Runs used to be trusted until a human noticed +// one of these by staring at the output. This module encodes the signatures +// so every multi-shard driver prints them and the campaign can refuse to +// publish flagged results. +// +// Severity: +// fatal — the run measured the wrong thing; do not publish, fix and rerun. +// warn — the run may be fine; re-check before publishing. + +import type { JitterResult } from "./stats.js"; + +export type ValiditySeverity = "fatal" | "warn"; + +export interface ValidityFlag { + severity: ValiditySeverity; + code: string; + message: string; +} + +export interface ValidityContext { + // Per-shard requested client count. Used to spot uniform shard ceilings. + perShardN?: number; + // Configured test duration; elapsed far beyond it means the runner's + // event loop stalled (a 30s test taking 94s was the whispers signature). + expectedDurationSec?: number; +} + +const round2 = (n: number) => Math.round(n * 100) / 100; + +// Checks that apply to a single JitterResult (per-shard or merged). +export function validateResult( + r: JitterResult, + ctx: ValidityContext = {}, +): ValidityFlag[] { + const flags: ValidityFlag[] = []; + + if (r.deliveryRatePct > 100.5) { + flags.push({ + severity: "fatal", + code: "delivery-over-100", + message: `delivery ${r.deliveryRatePct}% exceeds 100%: sender echo or duplicate counting (the Centrifugo 111.11% bug class)`, + }); + } + + if (r.clients > 0 && r.connectedClients === 0) { + flags.push({ + severity: "fatal", + code: "nothing-connected", + message: `0/${r.clients} clients connected: target down or unreachable, not a performance result`, + }); + } + + if (r.publishedMessages === 0 && r.clients > 0) { + flags.push({ + severity: "fatal", + code: "nothing-published", + message: + "publishedMessages=0 with clients connected: publisher misconfigured or silently 401'd (check ANYCABLE_BROADCAST_SECRET / BENCH_RUNNER_TOKEN on the shard)", + }); + } + + if (r.latencyOverMinMs.skewFloor < 0) { + flags.push({ + severity: "fatal", + code: "negative-skew-floor", + message: `skewFloor=${r.latencyOverMinMs.skewFloor}ms: sentAt and receivedAt came from different clocks; publish path is asymmetric`, + }); + } + + if ( + ctx.expectedDurationSec && + r.elapsedMs > ctx.expectedDurationSec * 1000 * 1.5 + ) { + flags.push({ + severity: "fatal", + code: "elapsed-overrun", + message: `elapsed ${round2(r.elapsedMs / 1000)}s vs configured ${ctx.expectedDurationSec}s: runner event loop saturated; latency and delivery are runner artifacts`, + }); + } + + if (r.connectFailures > 0 && r.connectFailures >= r.clients * 0.01) { + flags.push({ + severity: "warn", + code: "connect-failures", + message: `${r.connectFailures} connect failures (${round2((r.connectFailures / Math.max(1, r.clients)) * 100)}% of clients): check target health and shard port headroom`, + }); + } + + // Arithmetic cross-foot: received + lost should not exceed expected by + // more than rounding. A reconcile failure means the accounting is buggy, + // and every derived percentage inherits the bug. + if ( + r.expectedDeliveries > 0 && + r.receivedDeliveries > r.expectedDeliveries * 1.005 + ) { + flags.push({ + severity: "fatal", + code: "received-over-expected", + message: `receivedDeliveries ${r.receivedDeliveries} exceeds expected ${r.expectedDeliveries}: duplicate delivery counting`, + }); + } + + return flags; +} + +// Checks that only make sense across shards. +export function validateShardSet( + shardResults: JitterResult[], + ctx: ValidityContext = {}, +): ValidityFlag[] { + const flags: ValidityFlag[] = []; + if (shardResults.length < 2) return flags; + + // Uniform shard ceiling: every shard connected the same count, below what + // was asked of it. That is the load generator's wall (ephemeral ports, + // event loop), never the server's. The 600K idle cap (50 shards frozen at + // exactly 12,002) and the 132K capacity cap (13 runners at ~10K) both + // looked exactly like this. + const connected = shardResults.map((r) => r.connectedClients ?? r.clients); + const allEqual = connected.every((c) => c === connected[0]); + if ( + allEqual && + ctx.perShardN !== undefined && + connected[0] < ctx.perShardN * 0.99 + ) { + flags.push({ + severity: "fatal", + code: "uniform-shard-ceiling", + message: `every shard connected exactly ${connected[0]} of ${ctx.perShardN} requested: load-generator limit, not a server ceiling. Add shards or lower per-shard N; do not publish.`, + }); + } + + // Identical anomaly across shards with healthy connects but zero publishes + // on a subset: secret drift across the fleet. + const silent = shardResults.filter( + (r) => r.publishedMessages === 0 && (r.connectedClients ?? r.clients) > 0, + ); + if (silent.length > 0 && silent.length < shardResults.length) { + flags.push({ + severity: "fatal", + code: "partial-publisher-silence", + message: `${silent.length}/${shardResults.length} shards connected fine yet published nothing: secret missing on part of the fleet (the runners 14-50 bug class)`, + }); + } + + return flags; +} + +export function formatValidityReport(flags: ValidityFlag[]): string { + if (flags.length === 0) return "Validity: no flags."; + const lines = ["", "=== VALIDITY FLAGS ==="]; + for (const f of flags) { + lines.push(` [${f.severity.toUpperCase()}] ${f.code}: ${f.message}`); + } + const fatal = flags.filter((f) => f.severity === "fatal").length; + if (fatal > 0) { + lines.push( + ` ${fatal} fatal flag(s): this run measured the rig or a misconfig, not the server. Fix and rerun before publishing.`, + ); + } + return lines.join("\n"); +} + +export function hasFatal(flags: ValidityFlag[]): boolean { + return flags.some((f) => f.severity === "fatal"); +} diff --git a/backend/src/lib/idle-runner.ts b/backend/src/lib/idle-runner.ts index a9de5b7..d3427b9 100644 --- a/backend/src/lib/idle-runner.ts +++ b/backend/src/lib/idle-runner.ts @@ -108,6 +108,12 @@ export interface IdleParams { holdSec: number; rampPerSec: number; stream: string; + // AnyCable/Action Cable only. Channel to subscribe to (default "$pubsub" for + // the standalone anycable-go targets; "BenchmarkChannel" for a real Rails + // app) and the WebSocket subprotocol (extended "actioncable-v1-ext-json" for + // AnyCable, base "actioncable-v1-json" for vanilla Action Cable / Solid Cable). + channel?: string; + acProtocol?: string; } export interface IdleResult { @@ -135,8 +141,10 @@ export async function runIdleAnycable( const sockets: WebSocket[] = []; const startedAt = Date.now(); + const acProtocol = p.acProtocol ?? "actioncable-v1-ext-json"; + const channel = p.channel ?? "$pubsub"; for (let i = 0; i < p.n; i++) { - const ws = new WebSocket(cableUrl, ["actioncable-v1-ext-json"]); + const ws = new WebSocket(cableUrl, [acProtocol]); sockets.push(ws); // `failed` should count connection ATTEMPTS that never opened — @@ -156,13 +164,14 @@ export async function runIdleAnycable( const msg = JSON.parse(raw.toString()); if (msg.type === "welcome") { result.welcomed++; - // Subscribe to a $pubsub stream — works without RPC since - // anycable-go is started with ANYCABLE_PUBLIC=true. + // Subscribe. For "$pubsub" this works without RPC (anycable-go runs + // with ANYCABLE_PUBLIC=true); for a real Rails channel the gateway / + // Puma routes the subscribe command to the app. ws.send( JSON.stringify({ command: "subscribe", identifier: JSON.stringify({ - channel: "$pubsub", + channel, stream_name: p.stream, }), }) diff --git a/backend/src/lib/jitter-runners.ts b/backend/src/lib/jitter-runners.ts index d7870eb..c7dbe13 100644 --- a/backend/src/lib/jitter-runners.ts +++ b/backend/src/lib/jitter-runners.ts @@ -11,6 +11,14 @@ import WebSocket from "ws"; import { createCable } from "@anycable/core"; +import { + anycableDefaultStrategy, + makeTunedStrategy, + type ReconnectMode, +} from "./core/reconnect-strategies.js"; +// The official Rails client (for the Action Cable / Solid Cable / Async::Cable +// targets), set up to run under Node. AnyCable keeps @anycable/core. +import { ActionCable } from "./core/actioncable-node.js"; import { io as ioClient, Socket } from "socket.io-client"; import { ClientStat, JitterResult, newStat, recordMsg, summarize } from "./core/stats.js"; @@ -117,6 +125,32 @@ export interface AnycableUrls { cableUrl: string; broadcastUrl: string; broadcastSecret?: string; + // Channel to subscribe to. Defaults to "$pubsub" (anycable-go's public + // pub/sub channel, used by the standalone OSS/Pro targets via streamFrom). + // For a real Rails app, pass "BenchmarkChannel" and the driver subscribes + // to that named channel with { stream_name } params instead. + channel?: string; + // WebSocket subprotocol. AnyCable uses the extended Action Cable protocol + // ("actioncable-v1-ext-json") which carries the delivery-guarantee / + // resume machinery; vanilla Action Cable and Solid Cable speak the base + // protocol ("actioncable-v1-json"). + acProtocol?: string; + // Reconnect mode for the client's backoff: + // "default" (default) — each client's stock reconnect (what users get): + // @anycable/core → backoffWithJitter(3000) (~1.5–9s first reconnect); + // @rails/actioncable → its ConnectionMonitor (~6s first reconnect). + // "tuned" — a uniform aggressive-but-storm-safe backoff (~reconnectBaseMs + // first attempt) on BOTH clients, so the resume tail reflects the + // SERVER's replay speed rather than the client's stock delay. + // Run both and report side by side (default = real UX, tuned = server ceiling). + reconnectMode?: ReconnectMode; + // First-attempt base delay (ms) for the tuned profile. Default 500. + reconnectBaseMs?: number; + // Which JS client to drive with. "anycable" (default) = @anycable/core + // (extended protocol, resume) for the AnyCable target. "actioncable" = + // @rails/actioncable (the official Rails client, base protocol, no resume) + // for the Action Cable / Solid Cable / Async::Cable targets. + clientLib?: "anycable" | "actioncable"; } export async function runJitterAnycable( @@ -129,27 +163,120 @@ export async function runJitterAnycable( const rss = trackPeakRss(); const stats: ClientStat[] = []; - const cables: ReturnType[] = []; + // Unified control surface over the two client libraries so the jitter loop + // and teardown stay client-agnostic. disconnect()/connect() take the client + // cleanly offline and back — a standard fixed-length outage regardless of + // the client's own backoff. destroy() fully tears the client down at the end + // of the run (stops the reconnect monitor too), so no consumer keeps + // reconnecting inside the long-lived bench-runner process. + interface JitterConn { + disconnect(): void; + connect(): void; + destroy(): void; + } + const conns: JitterConn[] = []; + const useActionCable = urls.clientLib === "actioncable"; + const tuned = urls.reconnectMode === "tuned"; + const tunedBaseMs = urls.reconnectBaseMs && urls.reconnectBaseMs > 0 ? urls.reconnectBaseMs : 500; + + if (useActionCable) { + // @rails/actioncable reconnect timing lives on the ConnectionMonitor's + // static `staleThreshold` (seconds). It's a global in this long-lived + // process, so set it explicitly per run: tuned → fast (~tunedBaseMs), else + // restore the native 6 s default. reconnectionBackoffRate stays at Rails' + // 0.15 so only the base changes. + const CM = (ActionCable as unknown as { + ConnectionMonitor: { staleThreshold: number; reconnectionBackoffRate: number }; + }).ConnectionMonitor; + CM.staleThreshold = tuned ? tunedBaseMs / 1000 : 6; + CM.reconnectionBackoffRate = 0.15; + log.info(`[jitter-ac] actioncable reconnect: staleThreshold=${CM.staleThreshold}s (${tuned ? "tuned" : "default"})`); + } else { + log.info(`[jitter-ac] anycable reconnect: ${tuned ? `tuned base=${tunedBaseMs}ms` : "default backoffWithJitter(3000)"}`); + } for (let i = 0; i < p.n; i++) { const stat = newStat(); stats.push(stat); - const cable = createCable(urls.cableUrl, { - websocketImplementation: WebSocket as unknown as typeof globalThis.WebSocket, - protocol: "actioncable-v1-ext-json", - // The @anycable/core types don't include "error" yet; the runtime - // accepts any of error|warn|info|debug. - logLevel: "error" as never, - }); - cable.on("close", () => {}); - cable.on("disconnect", () => {}); - cable.on("connect", () => { - stat.everConnected = true; - }); - const channel = cable.streamFrom(p.stream); - channel.on("message", (msg: unknown) => recordMsg(stat, msg)); - cables.push(cable); + if (useActionCable) { + // Official Rails client. createConsumer connects lazily; the + // subscription re-establishes on reconnect (no resume, base protocol). + const consumer = ActionCable.createConsumer(urls.cableUrl); + consumer.subscriptions.create( + { channel: urls.channel ?? "BenchmarkChannel", stream_name: p.stream }, + { + connected() { + stat.everConnected = true; + }, + received(data: unknown) { + recordMsg(stat, data); + }, + } + ); + conns.push({ + // Drop the underlying socket uncleanly (like a network blip) but leave + // the ConnectionMonitor running, so the official Rails client recovers + // on its OWN native, poll-based schedule (seconds) rather than an + // immediate reconnect. This is what a real Action Cable app experiences. + disconnect: () => { + const conn = ( + consumer as unknown as { + connection?: { webSocket?: { close?: () => void } }; + } + ).connection; + conn?.webSocket?.close?.(); + }, + // No-op: the native monitor drives reconnection. + connect: () => {}, + // Full teardown: consumer.disconnect() also stops the ConnectionMonitor, + // so it does not keep reconnecting after the run (a bare socket close + // would leave the monitor polling and orphan a live consumer in the + // long-lived bench-runner process). + destroy: () => consumer.disconnect(), + }); + } else { + const cable = createCable(urls.cableUrl, { + websocketImplementation: WebSocket as unknown as typeof globalThis.WebSocket, + protocol: (urls.acProtocol ?? "actioncable-v1-ext-json") as never, + // The @anycable/core types don't include "error" yet; the runtime + // accepts any of error|warn|info|debug. + logLevel: "error" as never, + // Explicit named strategy (per PR review — no inline backoffWithJitter + // tweak): tuned → shared aggressive profile; default → @anycable/core's + // real stock backoff. + reconnectStrategy: tuned + ? makeTunedStrategy(tunedBaseMs) + : anycableDefaultStrategy, + }); + cable.on("close", () => {}); + cable.on("disconnect", () => {}); + cable.on("connect", () => { + stat.everConnected = true; + }); + // "$pubsub" -> anycable-go's signed pub/sub channel (streamFrom). Any + // other value -> a real Rails channel subscribed with { stream_name }. + const channel = + urls.channel && urls.channel !== "$pubsub" + ? cable.subscribeTo(urls.channel, { stream_name: p.stream }) + : cable.streamFrom(p.stream); + channel.on("message", (msg: unknown) => recordMsg(stat, msg)); + conns.push({ + // Unclean socket drop (terminate the underlying ws) so @anycable/core's + // Monitor sees an unexpected close and auto-reconnects on its OWN + // reconnectStrategy — the default backoff or the tuned profile. This is + // what makes the default-vs-tuned matrix real: a manual connect() would + // reconnect immediately and bypass the strategy entirely (default would + // read the same as tuned). connect() is a no-op; the Monitor drives + // recovery and AnyCable resumes missed messages (sid retained) once back. + disconnect: () => { + terminateCableWs(cable); + }, + connect: () => {}, + // @anycable/core's disconnect() stops its Monitor cleanly at teardown. + destroy: () => cable.disconnect(), + }); + } await maybePauseForRamp(p, i, "jitter-ac"); } @@ -166,28 +293,29 @@ export async function runJitterAnycable( }); const endAt = Date.now() + p.durationSec * 1000; - const jitterTasks = cables.map((cable, i) => + const jitterTasks = conns.map((conn, i) => (async () => { const stat = stats[i]; let next = Date.now() + (5 + Math.random() * p.jitterIntervalSec) * 1000; while (Date.now() < endAt) { if (Date.now() >= next) { - // Force-close the underlying TCP socket — same semantics as - // the Socket.io test (raw.terminate()). The cable's Monitor - // detects the close and reconnects with its built-in backoff, - // mirroring socket.io-client's retry path. Don't call - // cable.connect() manually — let the reconnect machinery run. - // - // Only count the jitter event when terminate actually severed - // a connection. If the cable is already mid-reconnect (no `ws` - // ref), we skip the count so csrResumeRatePct denominators stay - // honest. - if (terminateCableWs(cable)) { - stat.jitterCount++; - } - // Hold the "offline" window. Reconnect attempts may fire - // during or after this window — that's the system under test. + // Unclean network drop. BOTH clients then recover via their OWN + // reconnect machinery (connect() is a no-op), so each reflects real + // behavior and the default-vs-tuned reconnect profile actually bites: + // - @anycable/core: Monitor auto-reconnects on its reconnectStrategy + // (default backoff ~1.5-9s, or tuned ~0.5s), then AnyCable resumes + // messages broadcast during the outage (sid retained) → 100% + // delivery, tail = reconnect delay + replay. + // - @rails/actioncable: ConnectionMonitor reconnects on its + // staleThreshold (default ~6s, or tuned ~0.5s); base protocol has + // no resume, so messages in the window are lost → delivery reflects + // both the missing replay AND the real reconnect latency. + // The jitterDurationMs sleep just paces the loop before the next drop; + // actual offline time is set by each client's reconnect schedule. + stat.jitterCount++; + conn.disconnect(); await new Promise((r) => setTimeout(r, p.jitterDurationMs)); + conn.connect(); next = Date.now() + (p.jitterIntervalSec + Math.random() * 5) * 1000; } await new Promise((r) => setTimeout(r, 500)); @@ -197,9 +325,9 @@ export async function runJitterAnycable( await Promise.all([publishTask, ...jitterTasks]); - for (const c of cables) { + for (const conn of conns) { try { - c.disconnect(); + conn.destroy(); } catch { /* tear-down errors are not interesting */ } diff --git a/backend/src/lib/throughput.ts b/backend/src/lib/throughput.ts index cbb09e1..1a9201e 100644 --- a/backend/src/lib/throughput.ts +++ b/backend/src/lib/throughput.ts @@ -123,6 +123,14 @@ export interface AnycableUrls { // Optional NATS broadcaster — used when publisher mode is "nats". natsUrl?: string; // e.g. nats://anycable-go-pro.railway.internal:4242 natsSubject?: string; // default __anycable__ (matches anycable-go default) + // Channel to subscribe to. Defaults to "$pubsub" (anycable-go's public + // pub/sub channel via streamFrom). For a real Rails app, pass + // "BenchmarkChannel" and the driver subscribes to it with { stream_name }. + channel?: string; + // WebSocket subprotocol. AnyCable uses the extended Action Cable protocol + // ("actioncable-v1-ext-json"); vanilla Action Cable / Async::Cable speak the + // base protocol ("actioncable-v1-json"). + acProtocol?: string; } async function runAnycablePublisher( @@ -225,7 +233,7 @@ export async function runThroughputAnycable( stats.push(stat); const cable = createCable(urls.cableUrl, { websocketImplementation: WebSocket as unknown as typeof globalThis.WebSocket, - protocol: "actioncable-v1-ext-json", + protocol: (urls.acProtocol ?? "actioncable-v1-ext-json") as never, logLevel: "error" as never, }); cable.on("close", () => {}); @@ -233,7 +241,12 @@ export async function runThroughputAnycable( cable.on("connect", () => { stat.everConnected = true; }); - const channel = cable.streamFrom(p.stream); + // "$pubsub" -> anycable-go's signed pub/sub channel (streamFrom). Any + // other value -> a real Rails channel subscribed with { stream_name }. + const channel = + urls.channel && urls.channel !== "$pubsub" + ? cable.subscribeTo(urls.channel, { stream_name: p.stream }) + : cable.streamFrom(p.stream); channel.on("message", (msg: unknown) => recordMsg(stat, msg)); cables.push(cable); await maybePauseForRamp(p, i, "tp-ac"); diff --git a/backend/src/types/rails-actioncable.d.ts b/backend/src/types/rails-actioncable.d.ts new file mode 100644 index 0000000..0b7cb08 --- /dev/null +++ b/backend/src/types/rails-actioncable.d.ts @@ -0,0 +1,20 @@ +// Minimal ambient declaration for the official Rails Action Cable JS client. +// We only use createConsumer(...) and adapters.WebSocket in the bench runner. +declare module "@rails/actioncable" { + export const adapters: { WebSocket: unknown; logger: unknown }; + export interface Subscription { + unsubscribe(): void; + } + export interface Subscriptions { + create( + params: string | Record, + mixin?: Record, + ): Subscription; + } + export interface Consumer { + subscriptions: Subscriptions; + connect(): void; + disconnect(): void; + } + export function createConsumer(url?: string): Consumer; +} diff --git a/cable-bench-falcon/.dockerignore b/cable-bench-falcon/.dockerignore new file mode 100644 index 0000000..6751f09 --- /dev/null +++ b/cable-bench-falcon/.dockerignore @@ -0,0 +1,15 @@ +.git +.gitignore +# No encrypted credentials are used (SECRET_KEY_BASE_DUMMY=1 supplies the key). +# Copying master.key as a root-owned 0600 file breaks boot under USER rails. +/config/master.key +/log/* +/tmp/* +/storage/* +!/storage/.keep +/node_modules +/.bundle +/vendor/bundle +*.sqlite3 +*.sqlite3-* +.DS_Store diff --git a/cable-bench-falcon/.gitattributes b/cable-bench-falcon/.gitattributes new file mode 100644 index 0000000..8dc4323 --- /dev/null +++ b/cable-bench-falcon/.gitattributes @@ -0,0 +1,9 @@ +# See https://git-scm.com/docs/gitattributes for more about git attribute files. + +# Mark the database schema as having been generated. +db/schema.rb linguist-generated + +# Mark any vendored files as having been vendored. +vendor/* linguist-vendored +config/credentials/*.yml.enc diff=rails_credentials +config/credentials.yml.enc diff=rails_credentials diff --git a/cable-bench-falcon/.gitignore b/cable-bench-falcon/.gitignore new file mode 100644 index 0000000..fbcab40 --- /dev/null +++ b/cable-bench-falcon/.gitignore @@ -0,0 +1,35 @@ +# See https://help.github.com/articles/ignoring-files for more about ignoring files. +# +# Temporary files generated by your text editor or operating system +# belong in git's global ignore instead: +# `$XDG_CONFIG_HOME/git/ignore` or `~/.config/git/ignore` + +# Ignore bundler config. +/.bundle + +# Ignore all environment files. +/.env* + +# Ignore all logfiles and tempfiles. +/log/* +/tmp/* +!/log/.keep +!/tmp/.keep + +# Ignore pidfiles, but keep the directory. +/tmp/pids/* +!/tmp/pids/ +!/tmp/pids/.keep + +# Ignore storage (uploaded files in development and any SQLite databases). +/storage/* +!/storage/.keep +/tmp/storage/* +!/tmp/storage/ +!/tmp/storage/.keep + +/public/assets + +# Ignore key files for decrypting credentials and more. +/config/*.key + diff --git a/cable-bench-falcon/.rubocop.yml b/cable-bench-falcon/.rubocop.yml new file mode 100644 index 0000000..f9d86d4 --- /dev/null +++ b/cable-bench-falcon/.rubocop.yml @@ -0,0 +1,8 @@ +# Omakase Ruby styling for Rails +inherit_gem: { rubocop-rails-omakase: rubocop.yml } + +# Overwrite or add rules to create your own house style +# +# # Use `[a, [b, c]]` not `[ a, [ b, c ] ]` +# Layout/SpaceInsideArrayLiteralBrackets: +# Enabled: false diff --git a/cable-bench-falcon/.ruby-version b/cable-bench-falcon/.ruby-version new file mode 100644 index 0000000..f989260 --- /dev/null +++ b/cable-bench-falcon/.ruby-version @@ -0,0 +1 @@ +3.4.4 diff --git a/cable-bench-falcon/Dockerfile b/cable-bench-falcon/Dockerfile new file mode 100644 index 0000000..791b4e8 --- /dev/null +++ b/cable-bench-falcon/Dockerfile @@ -0,0 +1,36 @@ +# syntax=docker/dockerfile:1 +# Single image for the three Rails cable benchmark modes; bin/bench-entrypoint +# picks the adapter and process from BENCH_MODE. Mirrors the anycable-pro / +# socketioxide per-service pattern (railway.toml + --path-as-root cable-bench/). +ARG RUBY_VERSION=3.4.4 +FROM ruby:$RUBY_VERSION-slim AS base +WORKDIR /app +ENV RAILS_ENV=production \ + BUNDLE_DEPLOYMENT=1 \ + BUNDLE_WITHOUT=development:test \ + BUNDLE_PATH=/usr/local/bundle \ + RAILS_LOG_TO_STDOUT=1 \ + RAILS_SERVE_STATIC_FILES=1 \ + SECRET_KEY_BASE_DUMMY=1 +RUN apt-get update -qq && \ + apt-get install --no-install-recommends -y curl && \ + rm -rf /var/lib/apt/lists/* + +FROM base AS build +RUN apt-get update -qq && \ + apt-get install --no-install-recommends -y build-essential git pkg-config libyaml-dev libssl-dev && \ + rm -rf /var/lib/apt/lists/* +COPY Gemfile Gemfile.lock ./ +RUN bundle install && rm -rf "${BUNDLE_PATH}"/ruby/*/cache +COPY . . +RUN ./bin/rails assets:precompile + +FROM base +COPY --from=build /usr/local/bundle /usr/local/bundle +COPY --from=build /app /app +RUN useradd rails --create-home --shell /bin/bash && \ + mkdir -p storage tmp/pids log && \ + chown -R rails:rails db log storage tmp +USER rails +EXPOSE 3000 +ENTRYPOINT ["./bin/falcon-entrypoint"] diff --git a/cable-bench-falcon/Gemfile b/cable-bench-falcon/Gemfile new file mode 100644 index 0000000..3468427 --- /dev/null +++ b/cable-bench-falcon/Gemfile @@ -0,0 +1,71 @@ +source "https://rubygems.org" + +# actioncable-next must load before Rails / Action Cable so its load-path entry +# wins for `require "action_cable/..."`. It is the drop-in Action Cable fork +# (AnyCable) that adds the ActionCable::Server::Socket abstraction async-cable +# builds on. require:false — it is pulled in via the explicit action_cable +# requires in config/application.rb, not auto-required by name. +gem "actioncable-next", require: false + +# Bundle edge Rails instead: gem "rails", github: "rails/rails", branch: "main" +gem "rails", "~> 8.1.3" +# The modern asset pipeline for Rails [https://github.com/rails/propshaft] +gem "propshaft" +# Use sqlite3 as the database for Active Record +gem "sqlite3", ">= 2.1" + +# AsyncCable target: fiber-based Action Cable served in-process by Falcon +# (async ecosystem) instead of Puma's threads. Same Action Cable wire protocol; +# the WebSockets are handled by Async::Cable::Middleware on a Falcon reactor. +gem "falcon" +# Pinned to async-cable @27181dff1 (2026-05-29): the 0.3.1 release lacks +# Socket#raw_transmit, which actioncable-next fastlane broadcasts require +# (without it fastlane raises NoMethodError -> 0% delivery). This commit adds +# it natively and still allows Rails 8.1 (gemspec actioncable >= 8.1.0.alpha). +# We do NOT use main HEAD: the later Executor commit (dddef54c) bumped the +# gemspec to actioncable >= 8.2.0.alpha (edge Rails only). We instead vendor +# that fiber Executor in config/initializers/async_cable_executor.rb, since +# its code is self-contained and async-cable does not auto-wire it anyway. +gem "async-cable", github: "socketry/async-cable", ref: "27181dff124d2e5a933cb9abf2581c8e86532956", require: "async/cable" + +# Use JavaScript with ESM import maps [https://github.com/rails/importmap-rails] +gem "importmap-rails" +# Hotwire's SPA-like page accelerator [https://turbo.hotwired.dev] +gem "turbo-rails" +# Hotwire's modest JavaScript framework [https://stimulus.hotwired.dev] +gem "stimulus-rails" + +# Windows does not include zoneinfo files, so bundle the tzinfo-data gem +gem "tzinfo-data", platforms: %i[ windows jruby ] + +# Use the database-backed adapters for Rails.cache, Active Job, and Action Cable +gem "solid_cache" +gem "solid_queue" +gem "solid_cable" + +# Cross-process broadcast for the Falcon workers, matching the Action Cable +# Redis-adapter target so the only variable is the WS engine (Falcon fibers +# vs Puma threads). +gem "redis", ">= 4.0" + +# Reduces boot times through caching; required in config/boot.rb +gem "bootsnap", require: false + +group :development, :test do + # See https://guides.rubyonrails.org/debugging_rails_applications.html#debugging-with-the-debug-gem + gem "debug", platforms: %i[ mri windows ], require: "debug/prelude" + + # Audits gems for known security defects (use config/bundler-audit.yml to ignore issues) + gem "bundler-audit", require: false + + # Static analysis for security vulnerabilities [https://brakemanscanner.org/] + gem "brakeman", require: false + + # Omakase Ruby styling [https://github.com/rails/rubocop-rails-omakase/] + gem "rubocop-rails-omakase", require: false +end + +group :development do + # Use console on exceptions pages [https://github.com/rails/web-console] + gem "web-console" +end diff --git a/cable-bench-falcon/Gemfile.lock b/cable-bench-falcon/Gemfile.lock new file mode 100644 index 0000000..8e0ba1b --- /dev/null +++ b/cable-bench-falcon/Gemfile.lock @@ -0,0 +1,593 @@ +GIT + remote: https://github.com/socketry/async-cable.git + revision: 27181dff124d2e5a933cb9abf2581c8e86532956 + ref: 27181dff124d2e5a933cb9abf2581c8e86532956 + specs: + async-cable (0.3.1) + actioncable (>= 8.1.0.alpha) + async (~> 2.9) + async-websocket + +GEM + remote: https://rubygems.org/ + specs: + action_text-trix (2.1.19) + railties + actioncable (8.1.3) + actionpack (= 8.1.3) + activesupport (= 8.1.3) + nio4r (~> 2.0) + websocket-driver (>= 0.6.1) + zeitwerk (~> 2.6) + actioncable-next (0.3.4) + actionpack (>= 7.0, <= 8.2) + activesupport (>= 7.0, <= 8.2) + nio4r (~> 2.0) + websocket-driver (>= 0.6.1) + zeitwerk (~> 2.6) + actionmailbox (8.1.3) + actionpack (= 8.1.3) + activejob (= 8.1.3) + activerecord (= 8.1.3) + activestorage (= 8.1.3) + activesupport (= 8.1.3) + mail (>= 2.8.0) + actionmailer (8.1.3) + actionpack (= 8.1.3) + actionview (= 8.1.3) + activejob (= 8.1.3) + activesupport (= 8.1.3) + mail (>= 2.8.0) + rails-dom-testing (~> 2.2) + actionpack (8.1.3) + actionview (= 8.1.3) + activesupport (= 8.1.3) + nokogiri (>= 1.8.5) + rack (>= 2.2.4) + rack-session (>= 1.0.1) + rack-test (>= 0.6.3) + rails-dom-testing (~> 2.2) + rails-html-sanitizer (~> 1.6) + useragent (~> 0.16) + actiontext (8.1.3) + action_text-trix (~> 2.1.15) + actionpack (= 8.1.3) + activerecord (= 8.1.3) + activestorage (= 8.1.3) + activesupport (= 8.1.3) + globalid (>= 0.6.0) + nokogiri (>= 1.8.5) + actionview (8.1.3) + activesupport (= 8.1.3) + builder (~> 3.1) + erubi (~> 1.11) + rails-dom-testing (~> 2.2) + rails-html-sanitizer (~> 1.6) + activejob (8.1.3) + activesupport (= 8.1.3) + globalid (>= 0.3.6) + activemodel (8.1.3) + activesupport (= 8.1.3) + activerecord (8.1.3) + activemodel (= 8.1.3) + activesupport (= 8.1.3) + timeout (>= 0.4.0) + activestorage (8.1.3) + actionpack (= 8.1.3) + activejob (= 8.1.3) + activerecord (= 8.1.3) + activesupport (= 8.1.3) + marcel (~> 1.0) + activesupport (8.1.3) + base64 + bigdecimal + concurrent-ruby (~> 1.0, >= 1.3.1) + connection_pool (>= 2.2.5) + drb + i18n (>= 1.6, < 2) + json + logger (>= 1.4.2) + minitest (>= 5.1) + securerandom (>= 0.3) + tzinfo (~> 2.0, >= 2.0.5) + uri (>= 0.13.1) + ast (2.4.3) + async (2.40.0) + console (~> 1.29) + fiber-annotation + io-event (~> 1.11) + metrics (~> 0.12) + traces (~> 0.18) + async-container (0.37.0) + async (~> 2.22) + async-http (0.95.1) + async (>= 2.10.2) + async-pool (~> 0.11) + io-endpoint (~> 0.14) + io-stream (~> 0.6) + metrics (~> 0.12) + protocol-http (~> 0.62) + protocol-http1 (~> 0.39) + protocol-http2 (~> 0.26) + protocol-url (~> 0.2) + traces (~> 0.10) + async-http-cache (0.4.6) + async-http (~> 0.56) + async-pool (0.11.2) + async (>= 2.0) + async-service (0.24.1) + async + async-container (~> 0.34) + string-format (~> 0.2) + async-utilization (0.4.0) + console (~> 1.0) + async-websocket (0.30.1) + async-http (~> 0.76) + protocol-http (~> 0.34) + protocol-rack (~> 0.7) + protocol-websocket (~> 0.17) + bake (0.25.0) + bigdecimal + samovar (~> 2.1) + base64 (0.3.0) + bigdecimal (4.1.2) + bindex (0.8.1) + bootsnap (1.24.6) + msgpack (~> 1.2) + brakeman (8.0.5) + racc + builder (3.3.0) + bundler-audit (0.9.3) + bundler (>= 1.2.0) + thor (~> 1.0) + concurrent-ruby (1.3.7) + connection_pool (3.0.2) + console (1.36.0) + fiber-annotation + fiber-local (~> 1.1) + json + crass (1.0.7) + date (3.5.1) + debug (1.11.1) + irb (~> 1.10) + reline (>= 0.3.8) + drb (2.2.3) + erb (6.0.4) + erubi (1.13.1) + et-orbi (1.4.0) + tzinfo + falcon (0.55.5) + async + async-container (~> 0.20) + async-http (~> 0.75) + async-http-cache (~> 0.4) + async-service (~> 0.19) + async-utilization (~> 0.3) + bundler + localhost (~> 1.1) + openssl (>= 3.0) + protocol-http (~> 0.31) + protocol-rack (~> 0.7) + samovar (~> 2.3) + fiber-annotation (0.2.0) + fiber-local (1.1.0) + fiber-storage + fiber-storage (1.0.1) + fugit (1.12.2) + et-orbi (~> 1.4) + raabro (~> 1.4) + globalid (1.4.0) + activesupport (>= 6.1) + i18n (1.15.2) + concurrent-ruby (~> 1.0) + importmap-rails (2.2.3) + actionpack (>= 6.0.0) + activesupport (>= 6.0.0) + railties (>= 6.0.0) + io-console (0.8.2) + io-endpoint (0.17.2) + io-event (1.16.4) + io-stream (0.13.1) + irb (1.18.0) + pp (>= 0.6.0) + prism (>= 1.3.0) + rdoc (>= 4.0.0) + reline (>= 0.4.2) + json (2.20.0) + language_server-protocol (3.17.0.5) + lint_roller (1.1.0) + localhost (1.8.0) + bake + logger (1.7.0) + loofah (2.25.1) + crass (~> 1.0.2) + nokogiri (>= 1.12.0) + mail (2.9.0) + logger + mini_mime (>= 0.1.1) + net-imap + net-pop + net-smtp + marcel (1.2.1) + metrics (0.15.0) + mini_mime (1.1.5) + minitest (6.0.6) + drb (~> 2.0) + prism (~> 1.5) + msgpack (1.8.3) + net-imap (0.6.4.1) + date + net-protocol + net-pop (0.1.2) + net-protocol + net-protocol (0.2.2) + timeout + net-smtp (0.5.1) + net-protocol + nio4r (2.7.5) + nokogiri (1.19.4-aarch64-linux-gnu) + racc (~> 1.4) + nokogiri (1.19.4-aarch64-linux-musl) + racc (~> 1.4) + nokogiri (1.19.4-arm-linux-gnu) + racc (~> 1.4) + nokogiri (1.19.4-arm-linux-musl) + racc (~> 1.4) + nokogiri (1.19.4-arm64-darwin) + racc (~> 1.4) + nokogiri (1.19.4-x86_64-darwin) + racc (~> 1.4) + nokogiri (1.19.4-x86_64-linux-gnu) + racc (~> 1.4) + nokogiri (1.19.4-x86_64-linux-musl) + racc (~> 1.4) + openssl (4.0.2) + parallel (2.1.0) + parser (3.3.11.1) + ast (~> 2.4.1) + racc + pp (0.6.4) + prettyprint + prettyprint (0.2.0) + prism (1.9.0) + propshaft (1.3.2) + actionpack (>= 7.0.0) + activesupport (>= 7.0.0) + rack + protocol-hpack (1.5.1) + protocol-http (0.62.2) + protocol-http1 (0.39.0) + protocol-http (~> 0.62) + protocol-http2 (0.26.0) + protocol-hpack (~> 1.4) + protocol-http (~> 0.62) + protocol-rack (0.22.1) + io-stream (>= 0.10) + protocol-http (~> 0.58) + rack (>= 1.0) + protocol-url (0.4.0) + protocol-websocket (0.21.1) + protocol-http (~> 0.2) + raabro (1.4.0) + racc (1.8.1) + rack (3.2.6) + rack-session (2.1.2) + base64 (>= 0.1.0) + rack (>= 3.0.0) + rack-test (2.2.0) + rack (>= 1.3) + rackup (2.3.1) + rack (>= 3) + rails (8.1.3) + actioncable (= 8.1.3) + actionmailbox (= 8.1.3) + actionmailer (= 8.1.3) + actionpack (= 8.1.3) + actiontext (= 8.1.3) + actionview (= 8.1.3) + activejob (= 8.1.3) + activemodel (= 8.1.3) + activerecord (= 8.1.3) + activestorage (= 8.1.3) + activesupport (= 8.1.3) + bundler (>= 1.15.0) + railties (= 8.1.3) + rails-dom-testing (2.3.0) + activesupport (>= 5.0.0) + minitest + nokogiri (>= 1.6) + rails-html-sanitizer (1.7.0) + loofah (~> 2.25) + nokogiri (>= 1.15.7, != 1.16.7, != 1.16.6, != 1.16.5, != 1.16.4, != 1.16.3, != 1.16.2, != 1.16.1, != 1.16.0.rc1, != 1.16.0) + railties (8.1.3) + actionpack (= 8.1.3) + activesupport (= 8.1.3) + irb (~> 1.13) + rackup (>= 1.0.0) + rake (>= 12.2) + thor (~> 1.0, >= 1.2.2) + tsort (>= 0.2) + zeitwerk (~> 2.6) + rainbow (3.1.1) + rake (13.4.2) + rbs (4.0.3) + logger + prism (>= 1.6.0) + tsort + rdoc (8.0.0) + erb + prism (>= 1.6.0) + rbs (>= 4.0.0) + tsort + redis (5.4.1) + redis-client (>= 0.22.0) + redis-client (0.30.0) + connection_pool + regexp_parser (2.12.0) + reline (0.6.3) + io-console (~> 0.5) + rubocop (1.88.0) + json (~> 2.3) + language_server-protocol (~> 3.17.0.2) + lint_roller (~> 1.1.0) + parallel (>= 1.10) + parser (>= 3.3.0.2) + rainbow (>= 2.2.2, < 4.0) + regexp_parser (>= 2.9.3, < 3.0) + rubocop-ast (>= 1.49.0, < 2.0) + ruby-progressbar (~> 1.7) + unicode-display_width (>= 2.4.0, < 4.0) + rubocop-ast (1.49.1) + parser (>= 3.3.7.2) + prism (~> 1.7) + rubocop-performance (1.26.1) + lint_roller (~> 1.1) + rubocop (>= 1.75.0, < 2.0) + rubocop-ast (>= 1.47.1, < 2.0) + rubocop-rails (2.35.5) + activesupport (>= 4.2.0) + lint_roller (~> 1.1) + rack (>= 1.1) + rubocop (>= 1.75.0, < 2.0) + rubocop-ast (>= 1.44.0, < 2.0) + rubocop-rails-omakase (1.1.0) + rubocop (>= 1.72) + rubocop-performance (>= 1.24) + rubocop-rails (>= 2.30) + ruby-progressbar (1.13.0) + samovar (2.5.1) + console (~> 1.0) + securerandom (0.4.1) + solid_cable (4.0.0) + actioncable (>= 7.2) + activejob (>= 7.2) + activerecord (>= 7.2) + railties (>= 7.2) + solid_cache (1.0.10) + activejob (>= 7.2) + activerecord (>= 7.2) + railties (>= 7.2) + solid_queue (1.4.0) + activejob (>= 7.1) + activerecord (>= 7.1) + concurrent-ruby (>= 1.3.1) + fugit (~> 1.11) + railties (>= 7.1) + thor (>= 1.3.1) + sqlite3 (2.9.5-aarch64-linux-gnu) + sqlite3 (2.9.5-aarch64-linux-musl) + sqlite3 (2.9.5-arm-linux-gnu) + sqlite3 (2.9.5-arm-linux-musl) + sqlite3 (2.9.5-arm64-darwin) + sqlite3 (2.9.5-x86_64-darwin) + sqlite3 (2.9.5-x86_64-linux-gnu) + sqlite3 (2.9.5-x86_64-linux-musl) + stimulus-rails (1.3.4) + railties (>= 6.0.0) + string-format (0.2.0) + thor (1.5.0) + timeout (0.6.1) + traces (0.18.2) + tsort (0.2.0) + turbo-rails (2.0.23) + actionpack (>= 7.1.0) + railties (>= 7.1.0) + tzinfo (2.0.6) + concurrent-ruby (~> 1.0) + unicode-display_width (3.2.0) + unicode-emoji (~> 4.1) + unicode-emoji (4.2.0) + uri (1.1.1) + useragent (0.16.11) + web-console (4.3.0) + actionview (>= 8.0.0) + bindex (>= 0.4.0) + railties (>= 8.0.0) + websocket-driver (0.8.2) + base64 + websocket-extensions (>= 0.1.0) + websocket-extensions (0.1.5) + zeitwerk (2.8.2) + +PLATFORMS + aarch64-linux-gnu + aarch64-linux-musl + arm-linux-gnu + arm-linux-musl + arm64-darwin + x86_64-darwin + x86_64-linux-gnu + x86_64-linux-musl + +DEPENDENCIES + actioncable-next + async-cable! + bootsnap + brakeman + bundler-audit + debug + falcon + importmap-rails + propshaft + rails (~> 8.1.3) + redis (>= 4.0) + rubocop-rails-omakase + solid_cable + solid_cache + solid_queue + sqlite3 (>= 2.1) + stimulus-rails + turbo-rails + tzinfo-data + web-console + +CHECKSUMS + action_text-trix (2.1.19) sha256=7012f59421009cf284aa651294896414d653a61a2417c9b8714c8476d2f74009 + actioncable (8.1.3) sha256=e5bc7f75e44e6a22de29c4f43176927c3a9ce4824464b74ed18d8226e75a80f0 + actioncable-next (0.3.4) sha256=c4246f50c0c534500d18e6d0e49987aa33cbf3c598deb59e95e685182d9201ca + actionmailbox (8.1.3) sha256=df7da474eaa0e70df4ed5a6fef66eb3b3b0f2dbf7f14518deee8d77f1b4aae59 + actionmailer (8.1.3) sha256=831f724891bb70d0aaa4d76581a6321124b6a752cb655c9346aae5479318448d + actionpack (8.1.3) sha256=af998cae4d47c5d581a2cc363b5c77eb718b7c4b45748d81b1887b25621c29a3 + actiontext (8.1.3) sha256=d291019c00e1ea9e6463011fa214f6081a56d7b9a1d224e7d3f6384c1dafc7d2 + actionview (8.1.3) sha256=1347c88c7f3edb38100c5ce0e9fb5e62d7755f3edc1b61cce2eb0b2c6ea2fd5d + activejob (8.1.3) sha256=a149b1766aa8204c3c3da7309e4becd40fcd5529c348cffbf6c9b16b565fe8d3 + activemodel (8.1.3) sha256=90c05cbe4cef3649b8f79f13016191ea94c4525ce4a5c0fb7ef909c4b91c8219 + activerecord (8.1.3) sha256=8003be7b2466ba0a2a670e603eeb0a61dd66058fccecfc49901e775260ac70ab + activestorage (8.1.3) sha256=0564ce9309143951a67615e1bb4e090ee54b8befed417133cae614479b46384d + activesupport (8.1.3) sha256=21a5e0dfbd4c3ddd9e1317ec6a4d782fa226e7867dc70b0743acda81a1dca20e + ast (2.4.3) sha256=954615157c1d6a382bc27d690d973195e79db7f55e9765ac7c481c60bdb4d383 + async (2.40.0) sha256=52c7cf92b7e12fec4054f721b2fc9df401940d65a076f7e771cd6af18947af66 + async-cable (0.3.1) + async-container (0.37.0) sha256=16bd50a6d3ba818d917160a96a965a2cde0fd0e9959715741cbeeba90d68d315 + async-http (0.95.1) sha256=0c3dd458c204c06d5c4b20b01bbec4794a1203db627fb2ce536e1799ec14786c + async-http-cache (0.4.6) sha256=2038d1f093182f16b50b4db271c25085e3938da10bfcfc2904cadb0530fddfd6 + async-pool (0.11.2) sha256=0a43a17b02b04d9c451b7d12fafa9a50e55dc6dd00d4369aca00433f16a7e3ed + async-service (0.24.1) sha256=3ee313fa2d6c1427ffac68f42f272889c496fa1088cf1432b1e7a5f5bff56b06 + async-utilization (0.4.0) sha256=4da53cb1733a12c9cf70ff22bd37d29c10fa9162a03ddb10f34e12acd171fe32 + async-websocket (0.30.1) sha256=54bb8a8f184e4aa64434c7a78ecc55850a67a3d1dcd02e3ae787376e2c673936 + bake (0.25.0) sha256=a47bdc6a26addc048827debc36fe27bb4d5d71ac2958ad910b5386e9baf49869 + base64 (0.3.0) sha256=27337aeabad6ffae05c265c450490628ef3ebd4b67be58257393227588f5a97b + bigdecimal (4.1.2) sha256=53d217666027eab4280346fba98e7d5b66baaae1b9c3c1c0ffe89d48188a3fbd + bindex (0.8.1) sha256=7b1ecc9dc539ed8bccfc8cb4d2732046227b09d6f37582ff12e50a5047ceb17e + bootsnap (1.24.6) sha256=c60bab88c70332290f0a2636a288f675299eb4f804a02a3c085b42eca9da164a + brakeman (8.0.5) sha256=03735f9690d3fd4b32d66aacbf0a6d15a84266bdd06b32c05c8ecc8f6021d2be + builder (3.3.0) sha256=497918d2f9dca528fdca4b88d84e4ef4387256d984b8154e9d5d3fe5a9c8835f + bundler-audit (0.9.3) sha256=81c8766c71e47d0d28a0f98c7eed028539f21a6ea3cd8f685eb6f42333c9b4e9 + concurrent-ruby (1.3.7) sha256=4412caec3a5ea2e5fdc52076724c071a81f2c0593d83b2ac8cbb8ca63b3151b0 + connection_pool (3.0.2) sha256=33fff5ba71a12d2aa26cb72b1db8bba2a1a01823559fb01d29eb74c286e62e0a + console (1.36.0) sha256=45599ea906cf80a73d8941f03abf873fe66a6a954e0bac5bc1c01e2cdc406f07 + crass (1.0.7) sha256=94868719948664c89ddcaf0a37c65048413dfcb1c869470a5f7a7ceb5390b295 + date (3.5.1) sha256=750d06384d7b9c15d562c76291407d89e368dda4d4fff957eb94962d325a0dc0 + debug (1.11.1) sha256=2e0b0ac6119f2207a6f8ac7d4a73ca8eb4e440f64da0a3136c30343146e952b6 + drb (2.2.3) sha256=0b00d6fdb50995fe4a45dea13663493c841112e4068656854646f418fda13373 + erb (6.0.4) sha256=38e3803694be357fe2bfe312487c74beaf9fb4e5beb3e22498952fe1645b95d9 + erubi (1.13.1) sha256=a082103b0885dbc5ecf1172fede897f9ebdb745a4b97a5e8dc63953db1ee4ad9 + et-orbi (1.4.0) sha256=6c7e3c90779821f9e3b324c5e96fda9767f72995d6ae435b96678a4f3e2de8bc + falcon (0.55.5) sha256=8622db11361b6678578dd7e06b953aaef10e5006929ced618069bb62b0ff118d + fiber-annotation (0.2.0) sha256=7abfadf1d119f508867d4103bf231c0354d019cc39a5738945dec2edadaf6c03 + fiber-local (1.1.0) sha256=c885f94f210fb9b05737de65d511136ea602e00c5105953748aa0f8793489f06 + fiber-storage (1.0.1) sha256=f48e5b6d8b0be96dac486332b55cee82240057065dc761c1ea692b2e719240e1 + fugit (1.12.2) sha256=643f2bf28db263bd400cbf8e0dd8b76b2c9b94bdb130e12d2394de04d9c20e5e + globalid (1.4.0) sha256=037f12fbf1d9d7a014d501c2d5c77356fd4ddd96d7a7991d6700bba96706f427 + i18n (1.15.2) sha256=00f9eb62412fe593b2a65a97daa75300d37abb8f7202ec748e94b6d46a9dd1b5 + importmap-rails (2.2.3) sha256=7101be2a4dc97cf1558fb8f573a718404c5f6bcfe94f304bf1f39e444feeb16a + io-console (0.8.2) sha256=d6e3ae7a7cc7574f4b8893b4fca2162e57a825b223a177b7afa236c5ef9814cc + io-endpoint (0.17.2) sha256=3feaf766c116b35839c11fac68b6aaadc47887bb488902a57bf8e1d288fb3338 + io-event (1.16.4) sha256=98b04e3a5e374fe0ce20f69956d435a3430335920544e3c1a492bdf37c4bf6d6 + io-stream (0.13.1) sha256=570d7c4dfb0fbd767480b4a222048a2be6d9b78febc1ec68258d0e0a4cde20de + irb (1.18.0) sha256=de9454a0703a54704b9811a5ef31a60c86949fbf4013fcf244fabc7c775248e3 + json (2.20.0) sha256=9362bc6e55a952b056abf9167cf053358181c904cb70cd6eee0808ea830fc32b + language_server-protocol (3.17.0.5) sha256=fd1e39a51a28bf3eec959379985a72e296e9f9acfce46f6a79d31ca8760803cc + lint_roller (1.1.0) sha256=2c0c845b632a7d172cb849cc90c1bce937a28c5c8ccccb50dfd46a485003cc87 + localhost (1.8.0) sha256=df7ea825b4f64949c588c17efac86bc47ddc4460d723778abe933b71759b2701 + logger (1.7.0) sha256=196edec7cc44b66cfb40f9755ce11b392f21f7967696af15d274dde7edff0203 + loofah (2.25.1) sha256=d436c73dbd0c1147b16c4a41db097942d217303e1f7728704b37e4df9f6d2e04 + mail (2.9.0) sha256=6fa6673ecd71c60c2d996260f9ee3dd387d4673b8169b502134659ece6d34941 + marcel (1.2.1) sha256=1678e9360e32f9eafa917c80029e2f6d10b2715c66a4b87b6d0da9b9cd1f859f + metrics (0.15.0) sha256=61ded5bac95118e995b1bc9ed4a5f19bc9814928a312a85b200abbdac9039072 + mini_mime (1.1.5) sha256=8681b7e2e4215f2a159f9400b5816d85e9d8c6c6b491e96a12797e798f8bccef + minitest (6.0.6) sha256=153ea36d1d987a62942382b61075745042a2b3123b1cd48f4c3675af9cc7d6f1 + msgpack (1.8.3) sha256=8bda4a6428d3244e50d6bd55854d354edbada88a4e1f4f5731a39a0f86bee6a1 + net-imap (0.6.4.1) sha256=29f0360d75a7efd3539f16ac1957dea5c0a51ddeceb348db4553c3120914ea0d + net-pop (0.1.2) sha256=848b4e982013c15b2f0382792268763b748cce91c9e91e36b0f27ed26420dff3 + net-protocol (0.2.2) sha256=aa73e0cba6a125369de9837b8d8ef82a61849360eba0521900e2c3713aa162a8 + net-smtp (0.5.1) sha256=ed96a0af63c524fceb4b29b0d352195c30d82dd916a42f03c62a3a70e5b70736 + nio4r (2.7.5) sha256=6c90168e48fb5f8e768419c93abb94ba2b892a1d0602cb06eef16d8b7df1dca1 + nokogiri (1.19.4-aarch64-linux-gnu) sha256=1269fb644a6de405057a53dd5c762b1209b43ca7424f839454d3dbc677c31a8f + nokogiri (1.19.4-aarch64-linux-musl) sha256=35c65b9ce72b3bb03207bdbe7067915019dc18c1b9b59139684bd6690fdd01af + nokogiri (1.19.4-arm-linux-gnu) sha256=a301313e38bb065d68239e79734bcd6f56fb6efaacebde29e9abf2a4735340ca + nokogiri (1.19.4-arm-linux-musl) sha256=588923c101bcfa78869734d247d25b598674323e7f22474fc468f6e5647311eb + nokogiri (1.19.4-arm64-darwin) sha256=a46db9853286e6597b36ebc6953817d15acf3a299583eb3f89fdc6f91dd63527 + nokogiri (1.19.4-x86_64-darwin) sha256=7fd17057d3e1f00e9954a74b3cd76595d3d4a5ef233b7ed9599047c204f70551 + nokogiri (1.19.4-x86_64-linux-gnu) sha256=379fae440b28915e3f19d752ce2dcf8465ed2b2fbefd2a7ca0dd497bc981a06a + nokogiri (1.19.4-x86_64-linux-musl) sha256=17dfb7c1fa194ae02fbf7c51a7afc8d278045ab3fdacfd86f91d02d7b274470b + openssl (4.0.2) sha256=1037ad2868ae58df9ad917891c0c0f9815a1172f6846d4bcdd508e4c2ee747c2 + parallel (2.1.0) sha256=b35258865c2e31134c5ecb708beaaf6772adf9d5efae28e93e99260877b09356 + parser (3.3.11.1) sha256=d17ace7aabe3e72c3cc94043714be27cc6f852f104d81aa284c2281aecc65d54 + pp (0.6.4) sha256=dfcb0fce700c41456265922884f9fe195d7fbb0674a3578e6c0f69588e82b570 + prettyprint (0.2.0) sha256=2bc9e15581a94742064a3cc8b0fb9d45aae3d03a1baa6ef80922627a0766f193 + prism (1.9.0) sha256=7b530c6a9f92c24300014919c9dcbc055bf4cdf51ec30aed099b06cd6674ef85 + propshaft (1.3.2) sha256=1d56a3e56a92c21bfc29caf07406b5386b00d4c47ddf357cf989a5a234b1389e + protocol-hpack (1.5.1) sha256=6feca238b8078da1cd295677d6f306c6001af92d75fe0643d33e6956cbc3ad91 + protocol-http (0.62.2) sha256=e1c1f2f56029c1af8c4e2b8a67d0d096c76620f3afd8d99d1dcd2f6b8ffa773b + protocol-http1 (0.39.0) sha256=e49b3f4cda6f5d94c76a323d2b7f6977cba3ebd082d2da437039594da77ad8eb + protocol-http2 (0.26.0) sha256=bac89cd78082b241ccd0cf7246f5160e4bb0c9c975fb4bf7deef5f88cc317486 + protocol-rack (0.22.1) sha256=1185d245927ef9849a603700d6991ca353bc89724fbf98efa4a4333ed62a9fc3 + protocol-url (0.4.0) sha256=64d4c03b6b51ad815ac6fdaf77a1d91e5baf9220d26becb846c5459dacdea9e1 + protocol-websocket (0.21.1) sha256=34325e4325697f0956877e67784bcc838cfd51ebbf4f8e9e5201be292041ee61 + raabro (1.4.0) sha256=d4fa9ff5172391edb92b242eed8be802d1934b1464061ae5e70d80962c5da882 + racc (1.8.1) sha256=4a7f6929691dbec8b5209a0b373bc2614882b55fc5d2e447a21aaa691303d62f + rack (3.2.6) sha256=5ed78e1f73b2e25679bec7d45ee2d4483cc4146eb1be0264fc4d94cb5ef212c2 + rack-session (2.1.2) sha256=595434f8c0c3473ae7d7ac56ecda6cc6dfd9d37c0b2b5255330aa1576967ffe8 + rack-test (2.2.0) sha256=005a36692c306ac0b4a9350355ee080fd09ddef1148a5f8b2ac636c720f5c463 + rackup (2.3.1) sha256=6c79c26753778e90983761d677a48937ee3192b3ffef6bc963c0950f94688868 + rails (8.1.3) sha256=6d017ba5348c98fc909753a8169b21d44de14d2a0b92d140d1a966834c3c9cd3 + rails-dom-testing (2.3.0) sha256=8acc7953a7b911ca44588bf08737bc16719f431a1cc3091a292bca7317925c1d + rails-html-sanitizer (1.7.0) sha256=28b145cceaf9cc214a9874feaa183c3acba036c9592b19886e0e45efc62b1e89 + railties (8.1.3) sha256=913eb0e0cb520aac687ffd74916bd726d48fa21f47833c6292576ef6a286de22 + rainbow (3.1.1) sha256=039491aa3a89f42efa1d6dec2fc4e62ede96eb6acd95e52f1ad581182b79bc6a + rake (13.4.2) sha256=cb825b2bd5f1f8e91ca37bddb4b9aaf345551b4731da62949be002fa89283701 + rbs (4.0.3) sha256=5a7bf70e2628549d9a1f44eae447b2cfe55968a9c60cfff52693a4bdcc020e14 + rdoc (8.0.0) sha256=03bf8c08a9639658855a0cfd77c0abca8325c227693f7f33f82957811348c469 + redis (5.4.1) sha256=b5e675b57ad22b15c9bcc765d5ac26f60b675408af916d31527af9bd5a81faae + redis-client (0.30.0) sha256=743f11ed42f0a41a0341554087b077479fec7e2d47a7c123fd90a12c0db5e477 + regexp_parser (2.12.0) sha256=35a916a1d63190ab5c9009457136ae5f3c0c7512d60291d0d1378ba18ce08ebb + reline (0.6.3) sha256=1198b04973565b36ec0f11542ab3f5cfeeec34823f4e54cebde90968092b1835 + rubocop (1.88.0) sha256=e420ddf1662d0ef34bc8a2910ac4b396a7ddda0b51a708264405241734b08e0b + rubocop-ast (1.49.1) sha256=4412f3ee70f6fe4546cc489548e0f6fcf76cafcfa80fa03af67098ffed755035 + rubocop-performance (1.26.1) sha256=cd19b936ff196df85829d264b522fd4f98b6c89ad271fa52744a8c11b8f71834 + rubocop-rails (2.35.5) sha256=f00b3c936002ba8e9ac62e8607c54bb24cda44b36e41b9c7e4f3872e1b0f3fe3 + rubocop-rails-omakase (1.1.0) sha256=2af73ac8ee5852de2919abbd2618af9c15c19b512c4cfc1f9a5d3b6ef009109d + ruby-progressbar (1.13.0) sha256=80fc9c47a9b640d6834e0dc7b3c94c9df37f08cb072b7761e4a71e22cff29b33 + samovar (2.5.1) sha256=8a9fc41eb8868084f0321eb41678c485cfbc7d282fb306c0be67c3284b1d2394 + securerandom (0.4.1) sha256=cc5193d414a4341b6e225f0cb4446aceca8e50d5e1888743fac16987638ea0b1 + solid_cable (4.0.0) sha256=8379680ef6bf36e195eb876a6306ea290f87d5fa10bc4a757bc2a918f83229b5 + solid_cache (1.0.10) sha256=bc05a2fb3ac78a6f43cbb5946679cf9db67dd30d22939ededc385cb93e120d41 + solid_queue (1.4.0) sha256=e6a18d196f0b27cb6e3c77c5b31258b05fb634f8ed64fb1866ed164047216c2a + sqlite3 (2.9.5-aarch64-linux-gnu) sha256=78075b6337d3d182c6d2b4691049ed45cd220826160c9ea18946bf6a1de200dc + sqlite3 (2.9.5-aarch64-linux-musl) sha256=18c801185deb4adc01ddb281e8f672a39e3d1729979ca91e39439cd3eac0402d + sqlite3 (2.9.5-arm-linux-gnu) sha256=1bdfca0c7d63998c60b0f4a8e3c8df2d33800ccc4abd2d612eddbbbc92a4c48b + sqlite3 (2.9.5-arm-linux-musl) sha256=bae1109d12b2e9f588455967729b008e1ff4feb7761749df695019c9079913c6 + sqlite3 (2.9.5-arm64-darwin) sha256=d0cf444a70fc9395d513cfbcc1e6719e224aa645314e3824cb0474c721425aa2 + sqlite3 (2.9.5-x86_64-darwin) sha256=8e9caae38bd7ebb29cbeee3e7ab1d12dc2327d9a1b92c7fcf0dda05589627a81 + sqlite3 (2.9.5-x86_64-linux-gnu) sha256=233dbcb6714148dd23bc5aeb33e8efd6eac974969564ddd5794c23d5f52b231e + sqlite3 (2.9.5-x86_64-linux-musl) sha256=e7d3a7474e8af0f96150c21abc203fbab5437206bfcdf11deab7741c0ca516f2 + stimulus-rails (1.3.4) sha256=765676ffa1f33af64ce026d26b48e8ffb2e0b94e0f50e9119e11d6107d67cb06 + string-format (0.2.0) sha256=bc981c14116b061f12134549f32fa2d61a17b5a35dd6fd36596c21722a789af6 + thor (1.5.0) sha256=e3a9e55fe857e44859ce104a84675ab6e8cd59c650a49106a05f55f136425e73 + timeout (0.6.1) sha256=78f57368a7e7bbadec56971f78a3f5ecbcfb59b7fcbb0a3ed6ddc08a5094accb + traces (0.18.2) sha256=80f1649cb4daace1d7174b81f3b3b7427af0b93047759ba349960cb8f315e214 + tsort (0.2.0) sha256=9650a793f6859a43b6641671278f79cfead60ac714148aabe4e3f0060480089f + turbo-rails (2.0.23) sha256=ee0d90733aafff056cf51ff11e803d65e43cae258cc55f6492020ec1f9f9315f + tzinfo (2.0.6) sha256=8daf828cc77bcf7d63b0e3bdb6caa47e2272dcfaf4fbfe46f8c3a9df087a829b + unicode-display_width (3.2.0) sha256=0cdd96b5681a5949cdbc2c55e7b420facae74c4aaf9a9815eee1087cb1853c42 + unicode-emoji (4.2.0) sha256=519e69150f75652e40bf736106cfbc8f0f73aa3fb6a65afe62fefa7f80b0f80f + uri (1.1.1) sha256=379fa58d27ffb1387eaada68c749d1426738bd0f654d812fcc07e7568f5c57c6 + useragent (0.16.11) sha256=700e6413ad4bb954bb63547fa098dddf7b0ebe75b40cc6f93b8d54255b173844 + web-console (4.3.0) sha256=e13b71301cdfc2093f155b5aa3a622db80b4672d1f2f713119cc7ec7ac6a6da4 + websocket-driver (0.8.2) sha256=97c556b019bf3410b4961002ac501621e9322d3f8a7bc02161a09301cc4c4146 + websocket-extensions (0.1.5) sha256=1c6ba63092cda343eb53fc657110c71c754c56484aad42578495227d717a8241 + zeitwerk (2.8.2) sha256=7212a61311083c604184b1ea2574b9aa05cd14f855a0841c06985cabe9181d12 + +BUNDLED WITH + 4.0.4 diff --git a/cable-bench-falcon/README.md b/cable-bench-falcon/README.md new file mode 100644 index 0000000..7db80e4 --- /dev/null +++ b/cable-bench-falcon/README.md @@ -0,0 +1,24 @@ +# README + +This README would normally document whatever steps are necessary to get the +application up and running. + +Things you may want to cover: + +* Ruby version + +* System dependencies + +* Configuration + +* Database creation + +* Database initialization + +* How to run the test suite + +* Services (job queues, cache servers, search engines, etc.) + +* Deployment instructions + +* ... diff --git a/cable-bench-falcon/Rakefile b/cable-bench-falcon/Rakefile new file mode 100644 index 0000000..9a5ea73 --- /dev/null +++ b/cable-bench-falcon/Rakefile @@ -0,0 +1,6 @@ +# Add your own tasks in files placed in lib/tasks ending in .rake, +# for example lib/tasks/capistrano.rake, and they will automatically be available to Rake. + +require_relative "config/application" + +Rails.application.load_tasks diff --git a/cable-bench-falcon/app/assets/images/.keep b/cable-bench-falcon/app/assets/images/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench-falcon/app/assets/stylesheets/application.css b/cable-bench-falcon/app/assets/stylesheets/application.css new file mode 100644 index 0000000..fe93333 --- /dev/null +++ b/cable-bench-falcon/app/assets/stylesheets/application.css @@ -0,0 +1,10 @@ +/* + * This is a manifest file that'll be compiled into application.css. + * + * With Propshaft, assets are served efficiently without preprocessing steps. You can still include + * application-wide styles in this file, but keep in mind that CSS precedence will follow the standard + * cascading order, meaning styles declared later in the document or manifest will override earlier ones, + * depending on specificity. + * + * Consider organizing styles into separate files for maintainability. + */ diff --git a/cable-bench-falcon/app/channels/application_cable/channel.rb b/cable-bench-falcon/app/channels/application_cable/channel.rb new file mode 100644 index 0000000..d672697 --- /dev/null +++ b/cable-bench-falcon/app/channels/application_cable/channel.rb @@ -0,0 +1,4 @@ +module ApplicationCable + class Channel < ActionCable::Channel::Base + end +end diff --git a/cable-bench-falcon/app/channels/application_cable/connection.rb b/cable-bench-falcon/app/channels/application_cable/connection.rb new file mode 100644 index 0000000..333d36f --- /dev/null +++ b/cable-bench-falcon/app/channels/application_cable/connection.rb @@ -0,0 +1,7 @@ +module ApplicationCable + # The benchmark connects anonymous clients (the load generator opens raw + # WebSockets and subscribes to BenchmarkChannel). No app-level auth is + # performed, so every adapter is measured on the same handshake cost. + class Connection < ActionCable::Connection::Base + end +end diff --git a/cable-bench-falcon/app/channels/benchmark_channel.rb b/cable-bench-falcon/app/channels/benchmark_channel.rb new file mode 100644 index 0000000..c3b2ca2 --- /dev/null +++ b/cable-bench-falcon/app/channels/benchmark_channel.rb @@ -0,0 +1,16 @@ +# The one channel the harness subscribes to. A client subscribes with +# `{ channel: "BenchmarkChannel", stream_name: "" }` and receives every +# message broadcast to that stream. This is identical Action Cable code for +# all three adapters (Solid Cable, classic Action Cable / Redis, AnyCable) — +# what differs is the transport underneath, not the app. +class BenchmarkChannel < ApplicationCable::Channel + def subscribed + stream_from params[:stream_name] + end + + # Client-to-client fan-out used by the optional whispers test. Mirrors the + # Node socket.io server's whisper handler so the harness can reuse its driver. + def whisper(data) + ActionCable.server.broadcast(params[:stream_name], data) + end +end diff --git a/cable-bench-falcon/app/controllers/application_controller.rb b/cable-bench-falcon/app/controllers/application_controller.rb new file mode 100644 index 0000000..c353756 --- /dev/null +++ b/cable-bench-falcon/app/controllers/application_controller.rb @@ -0,0 +1,7 @@ +class ApplicationController < ActionController::Base + # Only allow modern browsers supporting webp images, web push, badges, import maps, CSS nesting, and CSS :has. + allow_browser versions: :modern + + # Changes to the importmap will invalidate the etag for HTML responses + stale_when_importmap_changes +end diff --git a/cable-bench-falcon/app/controllers/bench_controller.rb b/cable-bench-falcon/app/controllers/bench_controller.rb new file mode 100644 index 0000000..2ca8a13 --- /dev/null +++ b/cable-bench-falcon/app/controllers/bench_controller.rb @@ -0,0 +1,41 @@ +# The harness publishes through Rails so every adapter shares one publish +# path: POST /_bench/broadcast {stream, data} -> ActionCable.server.broadcast. +# For AnyCable this is patched to publish through the broker (extended +# protocol, reliable streams); for Action Cable / Solid Cable it goes through +# the Redis / DB adapter. Same code, different transport. +class BenchController < ActionController::Base + # The load generator posts JSON without a CSRF token. + skip_forgery_protection + + before_action :authorize, only: :broadcast + + # GET /health — liveness probe (open, no auth). + def health + render json: { status: "ok", mode: ENV.fetch("CABLE_ADAPTER", "solid_cable") } + end + + # POST /_bench/broadcast {stream, data} + # `data` arrives as a JSON string (the harness pre-serializes it); broadcast + # the parsed object so subscribers receive { seq, sentAt, text }. + def broadcast + payload = + begin + JSON.parse(params.require(:data)) + rescue JSON::ParserError, TypeError + params[:data] + end + ActionCable.server.broadcast(params.require(:stream), payload) + head :ok + end + + private + + # Optional bearer gate, matching the bench-runner's BENCH_RUNNER_TOKEN. If + # the env var is unset (local dev), the endpoint is open. + def authorize + expected = ENV["BENCH_RUNNER_TOKEN"] + return if expected.blank? + provided = request.headers["Authorization"].to_s.delete_prefix("Bearer ") + head :unauthorized unless ActiveSupport::SecurityUtils.secure_compare(provided, expected) + end +end diff --git a/cable-bench-falcon/app/controllers/concerns/.keep b/cable-bench-falcon/app/controllers/concerns/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench-falcon/app/helpers/application_helper.rb b/cable-bench-falcon/app/helpers/application_helper.rb new file mode 100644 index 0000000..de6be79 --- /dev/null +++ b/cable-bench-falcon/app/helpers/application_helper.rb @@ -0,0 +1,2 @@ +module ApplicationHelper +end diff --git a/cable-bench-falcon/app/javascript/application.js b/cable-bench-falcon/app/javascript/application.js new file mode 100644 index 0000000..0d7b494 --- /dev/null +++ b/cable-bench-falcon/app/javascript/application.js @@ -0,0 +1,3 @@ +// Configure your import map in config/importmap.rb. Read more: https://github.com/rails/importmap-rails +import "@hotwired/turbo-rails" +import "controllers" diff --git a/cable-bench-falcon/app/javascript/controllers/application.js b/cable-bench-falcon/app/javascript/controllers/application.js new file mode 100644 index 0000000..1213e85 --- /dev/null +++ b/cable-bench-falcon/app/javascript/controllers/application.js @@ -0,0 +1,9 @@ +import { Application } from "@hotwired/stimulus" + +const application = Application.start() + +// Configure Stimulus development experience +application.debug = false +window.Stimulus = application + +export { application } diff --git a/cable-bench-falcon/app/javascript/controllers/hello_controller.js b/cable-bench-falcon/app/javascript/controllers/hello_controller.js new file mode 100644 index 0000000..5975c07 --- /dev/null +++ b/cable-bench-falcon/app/javascript/controllers/hello_controller.js @@ -0,0 +1,7 @@ +import { Controller } from "@hotwired/stimulus" + +export default class extends Controller { + connect() { + this.element.textContent = "Hello World!" + } +} diff --git a/cable-bench-falcon/app/javascript/controllers/index.js b/cable-bench-falcon/app/javascript/controllers/index.js new file mode 100644 index 0000000..1156bf8 --- /dev/null +++ b/cable-bench-falcon/app/javascript/controllers/index.js @@ -0,0 +1,4 @@ +// Import and register all your controllers from the importmap via controllers/**/*_controller +import { application } from "controllers/application" +import { eagerLoadControllersFrom } from "@hotwired/stimulus-loading" +eagerLoadControllersFrom("controllers", application) diff --git a/cable-bench-falcon/app/jobs/application_job.rb b/cable-bench-falcon/app/jobs/application_job.rb new file mode 100644 index 0000000..d394c3d --- /dev/null +++ b/cable-bench-falcon/app/jobs/application_job.rb @@ -0,0 +1,7 @@ +class ApplicationJob < ActiveJob::Base + # Automatically retry jobs that encountered a deadlock + # retry_on ActiveRecord::Deadlocked + + # Most jobs are safe to ignore if the underlying records are no longer available + # discard_on ActiveJob::DeserializationError +end diff --git a/cable-bench-falcon/app/models/application_record.rb b/cable-bench-falcon/app/models/application_record.rb new file mode 100644 index 0000000..b63caeb --- /dev/null +++ b/cable-bench-falcon/app/models/application_record.rb @@ -0,0 +1,3 @@ +class ApplicationRecord < ActiveRecord::Base + primary_abstract_class +end diff --git a/cable-bench-falcon/app/models/concerns/.keep b/cable-bench-falcon/app/models/concerns/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench-falcon/app/views/layouts/application.html.erb b/cable-bench-falcon/app/views/layouts/application.html.erb new file mode 100644 index 0000000..95012f0 --- /dev/null +++ b/cable-bench-falcon/app/views/layouts/application.html.erb @@ -0,0 +1,29 @@ + + + + <%= content_for(:title) || "Cable Bench" %> + + + + + <%= csrf_meta_tags %> + <%= csp_meta_tag %> + + <%= yield :head %> + + <%# Enable PWA manifest for installable apps (make sure to enable in config/routes.rb too!) %> + <%#= tag.link rel: "manifest", href: pwa_manifest_path(format: :json) %> + + + + + + <%# Includes all stylesheet files in app/assets/stylesheets %> + <%= stylesheet_link_tag :app, "data-turbo-track": "reload" %> + <%= javascript_importmap_tags %> + + + + <%= yield %> + + diff --git a/cable-bench-falcon/app/views/pwa/manifest.json.erb b/cable-bench-falcon/app/views/pwa/manifest.json.erb new file mode 100644 index 0000000..9cbe4ae --- /dev/null +++ b/cable-bench-falcon/app/views/pwa/manifest.json.erb @@ -0,0 +1,22 @@ +{ + "name": "CableBench", + "icons": [ + { + "src": "/icon.png", + "type": "image/png", + "sizes": "512x512" + }, + { + "src": "/icon.png", + "type": "image/png", + "sizes": "512x512", + "purpose": "maskable" + } + ], + "start_url": "/", + "display": "standalone", + "scope": "/", + "description": "CableBench.", + "theme_color": "red", + "background_color": "red" +} diff --git a/cable-bench-falcon/app/views/pwa/service-worker.js b/cable-bench-falcon/app/views/pwa/service-worker.js new file mode 100644 index 0000000..b3a13fb --- /dev/null +++ b/cable-bench-falcon/app/views/pwa/service-worker.js @@ -0,0 +1,26 @@ +// Add a service worker for processing Web Push notifications: +// +// self.addEventListener("push", async (event) => { +// const { title, options } = await event.data.json() +// event.waitUntil(self.registration.showNotification(title, options)) +// }) +// +// self.addEventListener("notificationclick", function(event) { +// event.notification.close() +// event.waitUntil( +// clients.matchAll({ type: "window" }).then((clientList) => { +// for (let i = 0; i < clientList.length; i++) { +// let client = clientList[i] +// let clientPath = (new URL(client.url)).pathname +// +// if (clientPath == event.notification.data.path && "focus" in client) { +// return client.focus() +// } +// } +// +// if (clients.openWindow) { +// return clients.openWindow(event.notification.data.path) +// } +// }) +// ) +// }) diff --git a/cable-bench-falcon/bin/brakeman b/cable-bench-falcon/bin/brakeman new file mode 100755 index 0000000..ace1c9b --- /dev/null +++ b/cable-bench-falcon/bin/brakeman @@ -0,0 +1,7 @@ +#!/usr/bin/env ruby +require "rubygems" +require "bundler/setup" + +ARGV.unshift("--ensure-latest") + +load Gem.bin_path("brakeman", "brakeman") diff --git a/cable-bench-falcon/bin/bundler-audit b/cable-bench-falcon/bin/bundler-audit new file mode 100755 index 0000000..e2ef226 --- /dev/null +++ b/cable-bench-falcon/bin/bundler-audit @@ -0,0 +1,6 @@ +#!/usr/bin/env ruby +require_relative "../config/boot" +require "bundler/audit/cli" + +ARGV.concat %w[ --config config/bundler-audit.yml ] if ARGV.empty? || ARGV.include?("check") +Bundler::Audit::CLI.start diff --git a/cable-bench-falcon/bin/ci b/cable-bench-falcon/bin/ci new file mode 100755 index 0000000..4137ad5 --- /dev/null +++ b/cable-bench-falcon/bin/ci @@ -0,0 +1,6 @@ +#!/usr/bin/env ruby +require_relative "../config/boot" +require "active_support/continuous_integration" + +CI = ActiveSupport::ContinuousIntegration +require_relative "../config/ci.rb" diff --git a/cable-bench-falcon/bin/dev b/cable-bench-falcon/bin/dev new file mode 100755 index 0000000..5f91c20 --- /dev/null +++ b/cable-bench-falcon/bin/dev @@ -0,0 +1,2 @@ +#!/usr/bin/env ruby +exec "./bin/rails", "server", *ARGV diff --git a/cable-bench-falcon/bin/falcon-entrypoint b/cable-bench-falcon/bin/falcon-entrypoint new file mode 100755 index 0000000..46f7eaa --- /dev/null +++ b/cable-bench-falcon/bin/falcon-entrypoint @@ -0,0 +1,21 @@ +#!/bin/bash +# AsyncCable target: Rails served by Falcon (async/fiber reactor) instead of +# Puma. WebSockets at /cable are handled in-process by Async::Cable::Middleware; +# cross-process broadcast goes through the Redis cable adapter (matching the +# Action Cable Redis target so the only variable is the WS engine). +set -e + +export SECRET_KEY_BASE_DUMMY="${SECRET_KEY_BASE_DUMMY:-1}" +PORT="${PORT:-3000}" +# Falcon forks this many reactor processes. Match the Puma worker count used by +# the other targets (WEB_CONCURRENCY) so the box-level parallelism is the same. +COUNT="${WEB_CONCURRENCY:-8}" + +echo "[entrypoint] AsyncCable on Falcon: port=$PORT count=$COUNT adapter=redis" + +# Primary + solid_cache/solid_queue tables (cable fan-out is Redis, no DB). +./bin/rails db:prepare + +# Bind on IPv6 so Railway's private network can reach it. http:// = plaintext +# HTTP/1.1 + h2c; the WebSocket upgrade rides HTTP/1.1. +exec bundle exec falcon serve --bind "http://[::]:${PORT}" --count "${COUNT}" diff --git a/cable-bench-falcon/bin/importmap b/cable-bench-falcon/bin/importmap new file mode 100755 index 0000000..36502ab --- /dev/null +++ b/cable-bench-falcon/bin/importmap @@ -0,0 +1,4 @@ +#!/usr/bin/env ruby + +require_relative "../config/application" +require "importmap/commands" diff --git a/cable-bench-falcon/bin/jobs b/cable-bench-falcon/bin/jobs new file mode 100755 index 0000000..dcf59f3 --- /dev/null +++ b/cable-bench-falcon/bin/jobs @@ -0,0 +1,6 @@ +#!/usr/bin/env ruby + +require_relative "../config/environment" +require "solid_queue/cli" + +SolidQueue::Cli.start(ARGV) diff --git a/cable-bench-falcon/bin/rails b/cable-bench-falcon/bin/rails new file mode 100755 index 0000000..efc0377 --- /dev/null +++ b/cable-bench-falcon/bin/rails @@ -0,0 +1,4 @@ +#!/usr/bin/env ruby +APP_PATH = File.expand_path("../config/application", __dir__) +require_relative "../config/boot" +require "rails/commands" diff --git a/cable-bench-falcon/bin/rake b/cable-bench-falcon/bin/rake new file mode 100755 index 0000000..4fbf10b --- /dev/null +++ b/cable-bench-falcon/bin/rake @@ -0,0 +1,4 @@ +#!/usr/bin/env ruby +require_relative "../config/boot" +require "rake" +Rake.application.run diff --git a/cable-bench-falcon/bin/rubocop b/cable-bench-falcon/bin/rubocop new file mode 100755 index 0000000..5a20504 --- /dev/null +++ b/cable-bench-falcon/bin/rubocop @@ -0,0 +1,8 @@ +#!/usr/bin/env ruby +require "rubygems" +require "bundler/setup" + +# Explicit RuboCop config increases performance slightly while avoiding config confusion. +ARGV.unshift("--config", File.expand_path("../.rubocop.yml", __dir__)) + +load Gem.bin_path("rubocop", "rubocop") diff --git a/cable-bench-falcon/bin/setup b/cable-bench-falcon/bin/setup new file mode 100755 index 0000000..81be011 --- /dev/null +++ b/cable-bench-falcon/bin/setup @@ -0,0 +1,35 @@ +#!/usr/bin/env ruby +require "fileutils" + +APP_ROOT = File.expand_path("..", __dir__) + +def system!(*args) + system(*args, exception: true) +end + +FileUtils.chdir APP_ROOT do + # This script is a way to set up or update your development environment automatically. + # This script is idempotent, so that you can run it at any time and get an expectable outcome. + # Add necessary setup steps to this file. + + puts "== Installing dependencies ==" + system("bundle check") || system!("bundle install") + + # puts "\n== Copying sample files ==" + # unless File.exist?("config/database.yml") + # FileUtils.cp "config/database.yml.sample", "config/database.yml" + # end + + puts "\n== Preparing database ==" + system! "bin/rails db:prepare" + system! "bin/rails db:reset" if ARGV.include?("--reset") + + puts "\n== Removing old logs and tempfiles ==" + system! "bin/rails log:clear tmp:clear" + + unless ARGV.include?("--skip-server") + puts "\n== Starting development server ==" + STDOUT.flush # flush the output before exec(2) so that it displays + exec "bin/dev" + end +end diff --git a/cable-bench-falcon/bin/thrust b/cable-bench-falcon/bin/thrust new file mode 100755 index 0000000..36bde2d --- /dev/null +++ b/cable-bench-falcon/bin/thrust @@ -0,0 +1,5 @@ +#!/usr/bin/env ruby +require "rubygems" +require "bundler/setup" + +load Gem.bin_path("thruster", "thrust") diff --git a/cable-bench-falcon/config.ru b/cable-bench-falcon/config.ru new file mode 100644 index 0000000..4a3c09a --- /dev/null +++ b/cable-bench-falcon/config.ru @@ -0,0 +1,6 @@ +# This file is used by Rack-based servers to start the application. + +require_relative "config/environment" + +run Rails.application +Rails.application.load_server diff --git a/cable-bench-falcon/config/anycable.yml b/cable-bench-falcon/config/anycable.yml new file mode 100644 index 0000000..b2ebab2 --- /dev/null +++ b/cable-bench-falcon/config/anycable.yml @@ -0,0 +1,17 @@ +# AnyCable (RPC) configuration for the AnyCable benchmark mode only. The other +# two modes (solid_cable, redis) never load this — Rails serves /cable itself. +# +# Here Rails runs the gRPC RPC server (`bundle exec anycable`) and anycable-go +# is the separate WebSocket gateway. Broadcasts go over Redis Streams (redisx) +# so the gateway's broker can keep stream history and resume the extended +# Action Cable protocol on reconnect (the delivery-guarantee path). Wire the +# Redis URL and RPC bind via env in deployment (ANYCABLE_REDIS_URL, +# ANYCABLE_RPC_HOST). +default: &default + broadcast_adapter: redisx + +development: + <<: *default + +production: + <<: *default diff --git a/cable-bench-falcon/config/application.rb b/cable-bench-falcon/config/application.rb new file mode 100644 index 0000000..0aeaddf --- /dev/null +++ b/cable-bench-falcon/config/application.rb @@ -0,0 +1,42 @@ +require_relative "boot" + +require "rails" +# Pick the frameworks you want: +require "active_model/railtie" +require "active_job/railtie" +require "active_record/railtie" +# require "active_storage/engine" +require "action_controller/railtie" +# require "action_mailer/railtie" +# require "action_mailbox/engine" +# require "action_text/engine" +require "action_view/railtie" +require "action_cable/engine" +# require "rails/test_unit/railtie" + +# Require the gems listed in Gemfile, including any gems +# you've limited to :test, :development, or :production. +Bundler.require(*Rails.groups) + +module CableBench + class Application < Rails::Application + # Initialize configuration defaults for originally generated Rails version. + config.load_defaults 8.1 + + # Please, add to the `ignore` list any other `lib` subdirectories that do + # not contain `.rb` files, or that should not be reloaded or eager loaded. + # Common ones are `templates`, `generators`, or `middleware`, for example. + config.autoload_lib(ignore: %w[assets tasks]) + + # Configuration for the application, engines, and railties goes here. + # + # These settings can be overridden in specific environments using the files + # in config/environments, which are processed later. + # + # config.time_zone = "Central Time (US & Canada)" + # config.eager_load_paths << Rails.root.join("extras") + + # Don't generate system test files. + config.generators.system_tests = nil + end +end diff --git a/cable-bench-falcon/config/boot.rb b/cable-bench-falcon/config/boot.rb new file mode 100644 index 0000000..988a5dd --- /dev/null +++ b/cable-bench-falcon/config/boot.rb @@ -0,0 +1,4 @@ +ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__) + +require "bundler/setup" # Set up gems listed in the Gemfile. +require "bootsnap/setup" # Speed up boot time by caching expensive operations. diff --git a/cable-bench-falcon/config/bundler-audit.yml b/cable-bench-falcon/config/bundler-audit.yml new file mode 100644 index 0000000..e74b3af --- /dev/null +++ b/cable-bench-falcon/config/bundler-audit.yml @@ -0,0 +1,5 @@ +# Audit all gems listed in the Gemfile for known security problems by running bin/bundler-audit. +# CVEs that are not relevant to the application can be enumerated on the ignore list below. + +ignore: + - CVE-THAT-DOES-NOT-APPLY diff --git a/cable-bench-falcon/config/cable.yml b/cable-bench-falcon/config/cable.yml new file mode 100644 index 0000000..7877b67 --- /dev/null +++ b/cable-bench-falcon/config/cable.yml @@ -0,0 +1,15 @@ +# AsyncCable target. The WebSockets are served in-process by Falcon via +# Async::Cable::Middleware; this adapter is only the pub/sub fan-out path +# between Falcon worker processes. We use Redis to match the Action Cable +# Redis-adapter target exactly, so the only variable under test is the WS +# engine (Falcon fibers vs Puma threads). +development: + adapter: async + +test: + adapter: test + +production: + adapter: redis + url: <%= ENV.fetch("REDIS_URL", "redis://localhost:6379/1") %> + channel_prefix: cable_bench_falcon diff --git a/cable-bench-falcon/config/cache.yml b/cable-bench-falcon/config/cache.yml new file mode 100644 index 0000000..19d4908 --- /dev/null +++ b/cable-bench-falcon/config/cache.yml @@ -0,0 +1,16 @@ +default: &default + store_options: + # Cap age of oldest cache entry to fulfill retention policies + # max_age: <%= 60.days.to_i %> + max_size: <%= 256.megabytes %> + namespace: <%= Rails.env %> + +development: + <<: *default + +test: + <<: *default + +production: + database: cache + <<: *default diff --git a/cable-bench-falcon/config/ci.rb b/cable-bench-falcon/config/ci.rb new file mode 100644 index 0000000..239b343 --- /dev/null +++ b/cable-bench-falcon/config/ci.rb @@ -0,0 +1,20 @@ +# Run using bin/ci + +CI.run do + step "Setup", "bin/setup --skip-server" + + step "Style: Ruby", "bin/rubocop" + + step "Security: Gem audit", "bin/bundler-audit" + step "Security: Importmap vulnerability audit", "bin/importmap audit" + step "Security: Brakeman code analysis", "bin/brakeman --quiet --no-pager --exit-on-warn --exit-on-error" + + + # Optional: set a green GitHub commit status to unblock PR merge. + # Requires the `gh` CLI and `gh extension install basecamp/gh-signoff`. + # if success? + # step "Signoff: All systems go. Ready for merge and deploy.", "gh signoff" + # else + # failure "Signoff: CI failed. Do not merge or deploy.", "Fix the issues and try again." + # end +end diff --git a/cable-bench-falcon/config/credentials.yml.enc b/cable-bench-falcon/config/credentials.yml.enc new file mode 100644 index 0000000..1d889f4 --- /dev/null +++ b/cable-bench-falcon/config/credentials.yml.enc @@ -0,0 +1 @@ +eagm44xPbLb4jZgrmBiNBwIRQM+rbzjDEq8VpI1j0dSmvihIBuvdsO5hGvn88a9kr8ijIThnuR3el+OX7M8RLvfImeDVz6MgKj95zBAkHGwbi5CgMzjEd4jR2XyDdCxYpqi9WD1M3PAMymdHJpheREPMhOvF0CQ/Ac/57VKqiqOKybCSZGV7G30XOubAruk+kGKTdGEWgsMlTqhH+qbpDHP+ktn9Gu/v7LEoKkxCL1ETA6E53/6mwofWwcXuTClxn0oHkrOkm6NU4q6TfDBvTSPxuPVG600t29hPeBPt9RH/fa8ytsVpdbe7RUNobSr87bi7F5QudH0MOJBRezCou2LxeZgneokS7QJTh1/QhY572+UU3YePJhxyCukDIp07gewWOGPzz24/h/XNcyKLpUf9KIEGFQ7OQ++WcIIp+JSU/2yUCc+TiT0amnubSYOYjfzfPhKVydEYCkMOuH/2+ptJC8n7ETrp2L0v5pnXv732Cxnsgp3o2K96--5zE13fNJAxth31ZH--93XIYtVm7Aub41gBda9/Xw== \ No newline at end of file diff --git a/cable-bench-falcon/config/database.yml b/cable-bench-falcon/config/database.yml new file mode 100644 index 0000000..4e71461 --- /dev/null +++ b/cable-bench-falcon/config/database.yml @@ -0,0 +1,48 @@ +# SQLite. Versions 3.8.0 and up are supported. +# gem install sqlite3 +# +# Ensure the SQLite 3 gem is defined in your Gemfile +# gem "sqlite3" +# +default: &default + adapter: sqlite3 + max_connections: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %> + timeout: 5000 + +development: + <<: *default + database: storage/development.sqlite3 + +# Warning: The database defined as "test" will be erased and +# re-generated from your development database when you run "rake". +# Do not set this db to the same as development or production. +test: + <<: *default + database: storage/test.sqlite3 + +# SQLite3 write its data on the local filesystem, as such it requires +# persistent disks. If you are deploying to a managed service, you should +# make sure it provides disk persistence, as many don't. +# +# Similarly, if you deploy your application as a Docker container, you must +# ensure the database is located in a persisted volume. +# Benchmark data is disposable: the SQLite files live on the container's +# ephemeral disk and are recreated by `db:prepare` on every boot. The cable DB +# is the one that matters (Solid Cable's message stream); cache/queue are +# unused but kept so db:prepare succeeds. +production: + primary: + <<: *default + database: storage/production.sqlite3 + cache: + <<: *default + database: storage/production_cache.sqlite3 + migrations_paths: db/cache_migrate + queue: + <<: *default + database: storage/production_queue.sqlite3 + migrations_paths: db/queue_migrate + cable: + <<: *default + database: storage/production_cable.sqlite3 + migrations_paths: db/cable_migrate diff --git a/cable-bench-falcon/config/environment.rb b/cable-bench-falcon/config/environment.rb new file mode 100644 index 0000000..cac5315 --- /dev/null +++ b/cable-bench-falcon/config/environment.rb @@ -0,0 +1,5 @@ +# Load the Rails application. +require_relative "application" + +# Initialize the Rails application. +Rails.application.initialize! diff --git a/cable-bench-falcon/config/environments/development.rb b/cable-bench-falcon/config/environments/development.rb new file mode 100644 index 0000000..64dd9a0 --- /dev/null +++ b/cable-bench-falcon/config/environments/development.rb @@ -0,0 +1,71 @@ +require "active_support/core_ext/integer/time" + +Rails.application.configure do + # Settings specified here will take precedence over those in config/application.rb. + + # Make code changes take effect immediately without server restart. + config.enable_reloading = true + + # The load generator opens raw WebSockets with no Origin header (same as in + # production). Skip the Action Cable origin check so local smoke tests and + # the bench-runner can connect. + config.action_cable.disable_request_forgery_protection = true + + # Do not eager load code on boot. + config.eager_load = false + + # Show full error reports. + config.consider_all_requests_local = true + + # Enable server timing. + config.server_timing = true + + # Enable/disable Action Controller caching. By default Action Controller caching is disabled. + # Run rails dev:cache to toggle Action Controller caching. + if Rails.root.join("tmp/caching-dev.txt").exist? + config.action_controller.perform_caching = true + config.action_controller.enable_fragment_cache_logging = true + config.public_file_server.headers = { "cache-control" => "public, max-age=#{2.days.to_i}" } + else + config.action_controller.perform_caching = false + end + + # Change to :null_store to avoid any caching. + config.cache_store = :memory_store + + # Print deprecation notices to the Rails logger. + config.active_support.deprecation = :log + + # Raise an error on page load if there are pending migrations. + config.active_record.migration_error = :page_load + + # Highlight code that triggered database queries in logs. + config.active_record.verbose_query_logs = true + + # Append comments with runtime information tags to SQL queries in logs. + config.active_record.query_log_tags_enabled = true + + # Highlight code that enqueued background job in logs. + config.active_job.verbose_enqueue_logs = true + + # Highlight code that triggered redirect in logs. + config.action_dispatch.verbose_redirect_logs = true + + # Suppress logger output for asset requests. + config.assets.quiet = true + + # Raises error for missing translations. + # config.i18n.raise_on_missing_translations = true + + # Annotate rendered view with file names. + config.action_view.annotate_rendered_view_with_filenames = true + + # Uncomment if you wish to allow Action Cable access from any origin. + # config.action_cable.disable_request_forgery_protection = true + + # Raise error when a before_action's only/except options reference missing actions. + config.action_controller.raise_on_missing_callback_actions = true + + # Apply autocorrection by RuboCop to files generated by `bin/rails generate`. + # config.generators.apply_rubocop_autocorrect_after_generate! +end diff --git a/cable-bench-falcon/config/environments/production.rb b/cable-bench-falcon/config/environments/production.rb new file mode 100644 index 0000000..df867ef --- /dev/null +++ b/cable-bench-falcon/config/environments/production.rb @@ -0,0 +1,75 @@ +require "active_support/core_ext/integer/time" + +Rails.application.configure do + # Settings specified here will take precedence over those in config/application.rb. + + # Code is not reloaded between requests. + config.enable_reloading = false + + # Eager load code on boot for better performance and memory savings (ignored by Rake tasks). + config.eager_load = true + + # Full error reports are disabled. + config.consider_all_requests_local = false + + # Turn on fragment caching in view templates. + config.action_controller.perform_caching = true + + # Cache assets for far-future expiry since they are all digest stamped. + config.public_file_server.headers = { "cache-control" => "public, max-age=#{1.year.to_i}" } + + # Enable serving of images, stylesheets, and JavaScripts from an asset server. + # config.asset_host = "http://assets.example.com" + + # This app only ever receives traffic from the bench-runner fleet over + # Railway's private network (plain HTTP/WS, no TLS-terminating proxy in + # front), so SSL enforcement would just break internal connections. + config.assume_ssl = false + config.force_ssl = false + + # The load generator opens raw WebSockets without a matching Origin header; + # there is no browser session to protect, so skip the Action Cable origin check. + config.action_cable.disable_request_forgery_protection = true + + # Skip http-to-https redirect for the default health check endpoint. + # config.ssl_options = { redirect: { exclude: ->(request) { request.path == "/up" } } } + + # Log to STDOUT with the current request id as a default log tag. + config.log_tags = [ :request_id ] + config.logger = ActiveSupport::TaggedLogging.logger(STDOUT) + + # Change to "debug" to log everything (including potentially personally-identifiable information!). + config.log_level = ENV.fetch("RAILS_LOG_LEVEL", "info") + + # Prevent health checks from clogging up the logs. + config.silence_healthcheck_path = "/up" + + # Don't log any deprecations. + config.active_support.report_deprecations = false + + # Replace the default in-process memory cache store with a durable alternative. + config.cache_store = :solid_cache_store + + # Replace the default in-process and non-durable queuing backend for Active Job. + config.active_job.queue_adapter = :solid_queue + config.solid_queue.connects_to = { database: { writing: :queue } } + + # Enable locale fallbacks for I18n (makes lookups for any locale fall back to + # the I18n.default_locale when a translation cannot be found). + config.i18n.fallbacks = true + + # Do not dump schema after migrations. + config.active_record.dump_schema_after_migration = false + + # Only use :id for inspections in production. + config.active_record.attributes_for_inspect = [ :id ] + + # Enable DNS rebinding protection and other `Host` header attacks. + # config.hosts = [ + # "example.com", # Allow requests from example.com + # /.*\.example\.com/ # Allow requests from subdomains like `www.example.com` + # ] + # + # Skip DNS rebinding protection for the default health check endpoint. + # config.host_authorization = { exclude: ->(request) { request.path == "/up" } } +end diff --git a/cable-bench-falcon/config/environments/test.rb b/cable-bench-falcon/config/environments/test.rb new file mode 100644 index 0000000..14bc29e --- /dev/null +++ b/cable-bench-falcon/config/environments/test.rb @@ -0,0 +1,42 @@ +# The test environment is used exclusively to run your application's +# test suite. You never need to work with it otherwise. Remember that +# your test database is "scratch space" for the test suite and is wiped +# and recreated between test runs. Don't rely on the data there! + +Rails.application.configure do + # Settings specified here will take precedence over those in config/application.rb. + + # While tests run files are not watched, reloading is not necessary. + config.enable_reloading = false + + # Eager loading loads your entire application. When running a single test locally, + # this is usually not necessary, and can slow down your test suite. However, it's + # recommended that you enable it in continuous integration systems to ensure eager + # loading is working properly before deploying your code. + config.eager_load = ENV["CI"].present? + + # Configure public file server for tests with cache-control for performance. + config.public_file_server.headers = { "cache-control" => "public, max-age=3600" } + + # Show full error reports. + config.consider_all_requests_local = true + config.cache_store = :null_store + + # Render exception templates for rescuable exceptions and raise for other exceptions. + config.action_dispatch.show_exceptions = :rescuable + + # Disable request forgery protection in test environment. + config.action_controller.allow_forgery_protection = false + + # Print deprecation notices to the stderr. + config.active_support.deprecation = :stderr + + # Raises error for missing translations. + # config.i18n.raise_on_missing_translations = true + + # Annotate rendered view with file names. + # config.action_view.annotate_rendered_view_with_filenames = true + + # Raise error when a before_action's only/except options reference missing actions. + config.action_controller.raise_on_missing_callback_actions = true +end diff --git a/cable-bench-falcon/config/importmap.rb b/cable-bench-falcon/config/importmap.rb new file mode 100644 index 0000000..909dfc5 --- /dev/null +++ b/cable-bench-falcon/config/importmap.rb @@ -0,0 +1,7 @@ +# Pin npm packages by running ./bin/importmap + +pin "application" +pin "@hotwired/turbo-rails", to: "turbo.min.js" +pin "@hotwired/stimulus", to: "stimulus.min.js" +pin "@hotwired/stimulus-loading", to: "stimulus-loading.js" +pin_all_from "app/javascript/controllers", under: "controllers" diff --git a/cable-bench-falcon/config/initializers/action_cable.rb b/cable-bench-falcon/config/initializers/action_cable.rb new file mode 100644 index 0000000..f6dcc77 --- /dev/null +++ b/cable-bench-falcon/config/initializers/action_cable.rb @@ -0,0 +1,9 @@ +# actioncable-next supports "fastlane" broadcasts: a stream's payload is +# JSON-encoded once per channel identifier instead of once per subscriber, +# which roughly halves broadcast latency on large fan-outs. This is an +# actioncable-next optimization (stock Action Cable has no equivalent), so we +# enable it on the Async::Cable/Falcon target to measure that path. +# https://github.com/anycable/actioncable-next#actioncableserverconfigfastlane_broadcasts_enabled--true +Rails.application.config.after_initialize do + ActionCable.server.config.fastlane_broadcasts_enabled = true +end diff --git a/cable-bench-falcon/config/initializers/assets.rb b/cable-bench-falcon/config/initializers/assets.rb new file mode 100644 index 0000000..4873244 --- /dev/null +++ b/cable-bench-falcon/config/initializers/assets.rb @@ -0,0 +1,7 @@ +# Be sure to restart your server when you modify this file. + +# Version of your assets, change this if you want to expire all your assets. +Rails.application.config.assets.version = "1.0" + +# Add additional assets to the asset load path. +# Rails.application.config.assets.paths << Emoji.images_path diff --git a/cable-bench-falcon/config/initializers/async_cable_executor.rb b/cable-bench-falcon/config/initializers/async_cable_executor.rb new file mode 100644 index 0000000..96ee453 --- /dev/null +++ b/cable-bench-falcon/config/initializers/async_cable_executor.rb @@ -0,0 +1,159 @@ +# Install async-cable's fiber-based executor on this Falcon target. +# +# Action Cable dispatches pub/sub callback invocations and periodic timers +# through `ActionCable::Server::Base#executor`. Stock Rails (and the pinned +# async-cable @27181dff1) back this with a Concurrent::ThreadPoolExecutor, so +# under Falcon's fiber reactor every broadcast dispatch bounces through an OS +# thread before it reaches the socket. That thread hop is the prime suspect for +# the broadcast-latency gap vs Puma. +# +# Samuel Williams added `Async::Cable::Executor` (a fiber-based replacement) in +# async-cable commit dddef54c, but that same commit bumped the gemspec to +# `actioncable >= 8.2.0.alpha` (edge Rails only), and this app is Rails 8.1.3 + +# actioncable-next. The Executor code itself is self-contained (it only needs +# `async`) and async-cable does not auto-wire it anyway, so we vendor it here +# verbatim and install it by overriding the server's #executor. Vendored from +# https://github.com/socketry/async-cable/blob/dddef54c/lib/async/cable/executor.rb +require "async" + +module Async + module Cable + # Fiber-based replacement for `ActionCable::Server::ThreadedExecutor`. + # Tasks posted from inside a reactor run on the caller's reactor (no thread + # hop); tasks posted from outside, and recurring timers, run on a dedicated + # reactor thread owned by the executor. + class Executor + def initialize + @mutex = ::Thread::Mutex.new + @inbox = nil + @thread = nil + end + + def post(task = nil, &block) + block ||= task + + if current = ::Async::Task.current? + current.async { block.call } + else + inbox.push(proc { block.call }) + end + + return self + end + + def timer(interval, &block) + timer = Timer.new + + if current = ::Async::Task.current? + timer.task = current.async do |inner| + run_timer(inner, interval, block) + end + + return timer + end + + inbox = timer.inbox = self.inbox + begin + operation = proc do |task| + timer.task = task.async do |inner| + run_timer(inner, interval, block) + end + end + + inbox.push(operation) + rescue ::ClosedQueueError + # Executor is shutting down; match best-effort post-during-shutdown. + end + + return timer + end + + def shutdown + @mutex.synchronize do + return unless @thread + @inbox.close + @thread.join + @thread = nil + @inbox = nil + end + end + + class Timer + attr_writer :inbox + + def initialize + @inbox = nil + @mutex = ::Thread::Mutex.new + @task = nil + end + + def task=(task) + @mutex.synchronize { @task = task } + end + + def shutdown + task = nil + + @mutex.synchronize do + task = @task + @task = nil + end + return unless task + + if inbox = @inbox + begin + inbox.push(proc { task.stop }) + rescue ::ClosedQueueError + # Executor already shut down; timer stopped with its reactor. + end + else + task.stop + end + end + end + + private + + def inbox + @inbox || @mutex.synchronize { @inbox ||= start_thread } + end + + def run_timer(task, interval, block) + loop do + task.sleep(interval) + block.call + end + end + + def start_thread + inbox = ::Thread::Queue.new + + @thread = ::Thread.new do + ::Thread.current.name = "async-cable executor" + + Sync do |task| + while operation = inbox.pop + operation.call(task) + end + end + end + + return inbox + end + end + end +end + +# Override the Action Cable server's executor to use the fiber executor instead +# of the thread-pool ThreadedExecutor. Lazy + mutex-guarded, mirroring +# actioncable-next's own ActionCable::Server::Base#executor. +require "action_cable" +module AsyncCableFiberExecutor + # Reuses ActionCable::Server::Base's own @mutex/@executor ivars (set in its + # #initialize), matching actioncable-next's lazy #executor. If a future Rails + # bump stops initializing @mutex, this needs a guard. + def executor + @executor || @mutex.synchronize { @executor ||= Async::Cable::Executor.new } + end +end +ActionCable::Server::Base.prepend(AsyncCableFiberExecutor) diff --git a/cable-bench-falcon/config/initializers/content_security_policy.rb b/cable-bench-falcon/config/initializers/content_security_policy.rb new file mode 100644 index 0000000..d51d713 --- /dev/null +++ b/cable-bench-falcon/config/initializers/content_security_policy.rb @@ -0,0 +1,29 @@ +# Be sure to restart your server when you modify this file. + +# Define an application-wide content security policy. +# See the Securing Rails Applications Guide for more information: +# https://guides.rubyonrails.org/security.html#content-security-policy-header + +# Rails.application.configure do +# config.content_security_policy do |policy| +# policy.default_src :self, :https +# policy.font_src :self, :https, :data +# policy.img_src :self, :https, :data +# policy.object_src :none +# policy.script_src :self, :https +# policy.style_src :self, :https +# # Specify URI for violation reports +# # policy.report_uri "/csp-violation-report-endpoint" +# end +# +# # Generate session nonces for permitted importmap, inline scripts, and inline styles. +# config.content_security_policy_nonce_generator = ->(request) { request.session.id.to_s } +# config.content_security_policy_nonce_directives = %w(script-src style-src) +# +# # Automatically add `nonce` to `javascript_tag`, `javascript_include_tag`, and `stylesheet_link_tag` +# # if the corresponding directives are specified in `content_security_policy_nonce_directives`. +# # config.content_security_policy_nonce_auto = true +# +# # Report violations without enforcing the policy. +# # config.content_security_policy_report_only = true +# end diff --git a/cable-bench-falcon/config/initializers/filter_parameter_logging.rb b/cable-bench-falcon/config/initializers/filter_parameter_logging.rb new file mode 100644 index 0000000..c0b717f --- /dev/null +++ b/cable-bench-falcon/config/initializers/filter_parameter_logging.rb @@ -0,0 +1,8 @@ +# Be sure to restart your server when you modify this file. + +# Configure parameters to be partially matched (e.g. passw matches password) and filtered from the log file. +# Use this to limit dissemination of sensitive information. +# See the ActiveSupport::ParameterFilter documentation for supported notations and behaviors. +Rails.application.config.filter_parameters += [ + :passw, :email, :secret, :token, :_key, :crypt, :salt, :certificate, :otp, :ssn, :cvv, :cvc +] diff --git a/cable-bench-falcon/config/initializers/inflections.rb b/cable-bench-falcon/config/initializers/inflections.rb new file mode 100644 index 0000000..3860f65 --- /dev/null +++ b/cable-bench-falcon/config/initializers/inflections.rb @@ -0,0 +1,16 @@ +# Be sure to restart your server when you modify this file. + +# Add new inflection rules using the following format. Inflections +# are locale specific, and you may define rules for as many different +# locales as you wish. All of these examples are active by default: +# ActiveSupport::Inflector.inflections(:en) do |inflect| +# inflect.plural /^(ox)$/i, "\\1en" +# inflect.singular /^(ox)en/i, "\\1" +# inflect.irregular "person", "people" +# inflect.uncountable %w( fish sheep ) +# end + +# These inflection rules are supported but not enabled by default: +# ActiveSupport::Inflector.inflections(:en) do |inflect| +# inflect.acronym "RESTful" +# end diff --git a/cable-bench-falcon/config/locales/en.yml b/cable-bench-falcon/config/locales/en.yml new file mode 100644 index 0000000..6c349ae --- /dev/null +++ b/cable-bench-falcon/config/locales/en.yml @@ -0,0 +1,31 @@ +# Files in the config/locales directory are used for internationalization and +# are automatically loaded by Rails. If you want to use locales other than +# English, add the necessary files in this directory. +# +# To use the locales, use `I18n.t`: +# +# I18n.t "hello" +# +# In views, this is aliased to just `t`: +# +# <%= t("hello") %> +# +# To use a different locale, set it with `I18n.locale`: +# +# I18n.locale = :es +# +# This would use the information in config/locales/es.yml. +# +# To learn more about the API, please read the Rails Internationalization guide +# at https://guides.rubyonrails.org/i18n.html. +# +# Be aware that YAML interprets the following case-insensitive strings as +# booleans: `true`, `false`, `on`, `off`, `yes`, `no`. Therefore, these strings +# must be quoted to be interpreted as strings. For example: +# +# en: +# "yes": yup +# enabled: "ON" + +en: + hello: "Hello world" diff --git a/cable-bench-falcon/config/puma.rb b/cable-bench-falcon/config/puma.rb new file mode 100644 index 0000000..92ca455 --- /dev/null +++ b/cable-bench-falcon/config/puma.rb @@ -0,0 +1,45 @@ +# This configuration file will be evaluated by Puma. The top-level methods that +# are invoked here are part of Puma's configuration DSL. For more information +# about methods provided by the DSL, see https://puma.io/puma/Puma/DSL.html. +# +# Puma starts a configurable number of processes (workers) and each process +# serves each request in a thread from an internal thread pool. +# +# You can control the number of workers using ENV["WEB_CONCURRENCY"]. You +# should only set this value when you want to run 2 or more workers. The +# default is already 1. You can set it to `auto` to automatically start a worker +# for each available processor. +# +# The ideal number of threads per worker depends both on how much time the +# application spends waiting for IO operations and on how much you wish to +# prioritize throughput over latency. +# +# As a rule of thumb, increasing the number of threads will increase how much +# traffic a given process can handle (throughput), but due to CRuby's +# Global VM Lock (GVL) it has diminishing returns and will degrade the +# response time (latency) of the application. +# +# The default is set to 3 threads as it's deemed a decent compromise between +# throughput and latency for the average Rails application. +# +# Any libraries that use a connection pool or another resource pool should +# be configured to provide at least as many connections as the number of +# threads. This includes Active Record's `pool` parameter in `database.yml`. +threads_count = ENV.fetch("RAILS_MAX_THREADS", 3) +threads threads_count, threads_count + +# Bind IPv6 dual-stack (`[::]`) rather than the default IPv4 `0.0.0.0`: Railway's +# private network routes over IPv6, so internal clients (the bench-runner fleet) +# can only reach Puma on `[::]`. On Linux this also accepts IPv4, and it works +# locally on macOS too. A single bind avoids an address-in-use clash with `port`. +bind "tcp://[::]:#{ENV.fetch('PORT', 3000)}" + +# Allow puma to be restarted by `bin/rails restart` command. +plugin :tmp_restart + +# Run the Solid Queue supervisor inside of Puma for single-server deployments. +plugin :solid_queue if ENV["SOLID_QUEUE_IN_PUMA"] + +# Specify the PID file. Defaults to tmp/pids/server.pid in development. +# In other environments, only set the PID file if requested. +pidfile ENV["PIDFILE"] if ENV["PIDFILE"] diff --git a/cable-bench-falcon/config/queue.yml b/cable-bench-falcon/config/queue.yml new file mode 100644 index 0000000..9eace59 --- /dev/null +++ b/cable-bench-falcon/config/queue.yml @@ -0,0 +1,18 @@ +default: &default + dispatchers: + - polling_interval: 1 + batch_size: 500 + workers: + - queues: "*" + threads: 3 + processes: <%= ENV.fetch("JOB_CONCURRENCY", 1) %> + polling_interval: 0.1 + +development: + <<: *default + +test: + <<: *default + +production: + <<: *default diff --git a/cable-bench-falcon/config/recurring.yml b/cable-bench-falcon/config/recurring.yml new file mode 100644 index 0000000..b4207f9 --- /dev/null +++ b/cable-bench-falcon/config/recurring.yml @@ -0,0 +1,15 @@ +# examples: +# periodic_cleanup: +# class: CleanSoftDeletedRecordsJob +# queue: background +# args: [ 1000, { batch_size: 500 } ] +# schedule: every hour +# periodic_cleanup_with_command: +# command: "SoftDeletedRecord.due.delete_all" +# priority: 2 +# schedule: at 5am every day + +production: + clear_solid_queue_finished_jobs: + command: "SolidQueue::Job.clear_finished_in_batches(sleep_between_batches: 0.3)" + schedule: every hour at minute 12 diff --git a/cable-bench-falcon/config/routes.rb b/cable-bench-falcon/config/routes.rb new file mode 100644 index 0000000..b0923ee --- /dev/null +++ b/cable-bench-falcon/config/routes.rb @@ -0,0 +1,22 @@ +Rails.application.routes.draw do + # Define your application routes per the DSL in https://guides.rubyonrails.org/routing.html + + # Reveal health status on /up that returns 200 if the app boots with no exceptions, otherwise 500. + # Can be used by load balancers and uptime monitors to verify that the app is live. + get "up" => "rails/health#show", as: :rails_health_check + + # Benchmark harness surface. + get "health" => "bench#health" + post "_bench/broadcast" => "bench#broadcast" + + # WebSockets at /cable are handled by Async::Cable::Middleware (inserted by + # the async-cable railtie), which intercepts the WS upgrade before the router + # and dispatches to ActionCable on the Falcon reactor. No explicit mount. + + # Render dynamic PWA files from app/views/pwa/* (remember to link manifest in application.html.erb) + # get "manifest" => "rails/pwa#manifest", as: :pwa_manifest + # get "service-worker" => "rails/pwa#service_worker", as: :pwa_service_worker + + # Defines the root path route ("/") + # root "posts#index" +end diff --git a/cable-bench-falcon/db/cable_schema.rb b/cable-bench-falcon/db/cable_schema.rb new file mode 100644 index 0000000..2366660 --- /dev/null +++ b/cable-bench-falcon/db/cable_schema.rb @@ -0,0 +1,11 @@ +ActiveRecord::Schema[7.1].define(version: 1) do + create_table "solid_cable_messages", force: :cascade do |t| + t.binary "channel", limit: 1024, null: false + t.binary "payload", limit: 536870912, null: false + t.datetime "created_at", null: false + t.integer "channel_hash", limit: 8, null: false + t.index ["channel"], name: "index_solid_cable_messages_on_channel" + t.index ["channel_hash"], name: "index_solid_cable_messages_on_channel_hash" + t.index ["created_at"], name: "index_solid_cable_messages_on_created_at" + end +end diff --git a/cable-bench-falcon/db/cache_schema.rb b/cable-bench-falcon/db/cache_schema.rb new file mode 100644 index 0000000..81a410d --- /dev/null +++ b/cable-bench-falcon/db/cache_schema.rb @@ -0,0 +1,12 @@ +ActiveRecord::Schema[7.2].define(version: 1) do + create_table "solid_cache_entries", force: :cascade do |t| + t.binary "key", limit: 1024, null: false + t.binary "value", limit: 536870912, null: false + t.datetime "created_at", null: false + t.integer "key_hash", limit: 8, null: false + t.integer "byte_size", limit: 4, null: false + t.index ["byte_size"], name: "index_solid_cache_entries_on_byte_size" + t.index ["key_hash", "byte_size"], name: "index_solid_cache_entries_on_key_hash_and_byte_size" + t.index ["key_hash"], name: "index_solid_cache_entries_on_key_hash", unique: true + end +end diff --git a/cable-bench-falcon/db/queue_schema.rb b/cable-bench-falcon/db/queue_schema.rb new file mode 100644 index 0000000..85194b6 --- /dev/null +++ b/cable-bench-falcon/db/queue_schema.rb @@ -0,0 +1,129 @@ +ActiveRecord::Schema[7.1].define(version: 1) do + create_table "solid_queue_blocked_executions", force: :cascade do |t| + t.bigint "job_id", null: false + t.string "queue_name", null: false + t.integer "priority", default: 0, null: false + t.string "concurrency_key", null: false + t.datetime "expires_at", null: false + t.datetime "created_at", null: false + t.index [ "concurrency_key", "priority", "job_id" ], name: "index_solid_queue_blocked_executions_for_release" + t.index [ "expires_at", "concurrency_key" ], name: "index_solid_queue_blocked_executions_for_maintenance" + t.index [ "job_id" ], name: "index_solid_queue_blocked_executions_on_job_id", unique: true + end + + create_table "solid_queue_claimed_executions", force: :cascade do |t| + t.bigint "job_id", null: false + t.bigint "process_id" + t.datetime "created_at", null: false + t.index [ "job_id" ], name: "index_solid_queue_claimed_executions_on_job_id", unique: true + t.index [ "process_id", "job_id" ], name: "index_solid_queue_claimed_executions_on_process_id_and_job_id" + end + + create_table "solid_queue_failed_executions", force: :cascade do |t| + t.bigint "job_id", null: false + t.text "error" + t.datetime "created_at", null: false + t.index [ "job_id" ], name: "index_solid_queue_failed_executions_on_job_id", unique: true + end + + create_table "solid_queue_jobs", force: :cascade do |t| + t.string "queue_name", null: false + t.string "class_name", null: false + t.text "arguments" + t.integer "priority", default: 0, null: false + t.string "active_job_id" + t.datetime "scheduled_at" + t.datetime "finished_at" + t.string "concurrency_key" + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + t.index [ "active_job_id" ], name: "index_solid_queue_jobs_on_active_job_id" + t.index [ "class_name" ], name: "index_solid_queue_jobs_on_class_name" + t.index [ "finished_at" ], name: "index_solid_queue_jobs_on_finished_at" + t.index [ "queue_name", "finished_at" ], name: "index_solid_queue_jobs_for_filtering" + t.index [ "scheduled_at", "finished_at" ], name: "index_solid_queue_jobs_for_alerting" + end + + create_table "solid_queue_pauses", force: :cascade do |t| + t.string "queue_name", null: false + t.datetime "created_at", null: false + t.index [ "queue_name" ], name: "index_solid_queue_pauses_on_queue_name", unique: true + end + + create_table "solid_queue_processes", force: :cascade do |t| + t.string "kind", null: false + t.datetime "last_heartbeat_at", null: false + t.bigint "supervisor_id" + t.integer "pid", null: false + t.string "hostname" + t.text "metadata" + t.datetime "created_at", null: false + t.string "name", null: false + t.index [ "last_heartbeat_at" ], name: "index_solid_queue_processes_on_last_heartbeat_at" + t.index [ "name", "supervisor_id" ], name: "index_solid_queue_processes_on_name_and_supervisor_id", unique: true + t.index [ "supervisor_id" ], name: "index_solid_queue_processes_on_supervisor_id" + end + + create_table "solid_queue_ready_executions", force: :cascade do |t| + t.bigint "job_id", null: false + t.string "queue_name", null: false + t.integer "priority", default: 0, null: false + t.datetime "created_at", null: false + t.index [ "job_id" ], name: "index_solid_queue_ready_executions_on_job_id", unique: true + t.index [ "priority", "job_id" ], name: "index_solid_queue_poll_all" + t.index [ "queue_name", "priority", "job_id" ], name: "index_solid_queue_poll_by_queue" + end + + create_table "solid_queue_recurring_executions", force: :cascade do |t| + t.bigint "job_id", null: false + t.string "task_key", null: false + t.datetime "run_at", null: false + t.datetime "created_at", null: false + t.index [ "job_id" ], name: "index_solid_queue_recurring_executions_on_job_id", unique: true + t.index [ "task_key", "run_at" ], name: "index_solid_queue_recurring_executions_on_task_key_and_run_at", unique: true + end + + create_table "solid_queue_recurring_tasks", force: :cascade do |t| + t.string "key", null: false + t.string "schedule", null: false + t.string "command", limit: 2048 + t.string "class_name" + t.text "arguments" + t.string "queue_name" + t.integer "priority", default: 0 + t.boolean "static", default: true, null: false + t.text "description" + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + t.index [ "key" ], name: "index_solid_queue_recurring_tasks_on_key", unique: true + t.index [ "static" ], name: "index_solid_queue_recurring_tasks_on_static" + end + + create_table "solid_queue_scheduled_executions", force: :cascade do |t| + t.bigint "job_id", null: false + t.string "queue_name", null: false + t.integer "priority", default: 0, null: false + t.datetime "scheduled_at", null: false + t.datetime "created_at", null: false + t.index [ "job_id" ], name: "index_solid_queue_scheduled_executions_on_job_id", unique: true + t.index [ "scheduled_at", "priority", "job_id" ], name: "index_solid_queue_dispatch_all" + end + + create_table "solid_queue_semaphores", force: :cascade do |t| + t.string "key", null: false + t.integer "value", default: 1, null: false + t.datetime "expires_at", null: false + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + t.index [ "expires_at" ], name: "index_solid_queue_semaphores_on_expires_at" + t.index [ "key", "value" ], name: "index_solid_queue_semaphores_on_key_and_value" + t.index [ "key" ], name: "index_solid_queue_semaphores_on_key", unique: true + end + + add_foreign_key "solid_queue_blocked_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade + add_foreign_key "solid_queue_claimed_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade + add_foreign_key "solid_queue_failed_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade + add_foreign_key "solid_queue_ready_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade + add_foreign_key "solid_queue_recurring_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade + add_foreign_key "solid_queue_scheduled_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade +end diff --git a/cable-bench-falcon/db/seeds.rb b/cable-bench-falcon/db/seeds.rb new file mode 100644 index 0000000..4fbd6ed --- /dev/null +++ b/cable-bench-falcon/db/seeds.rb @@ -0,0 +1,9 @@ +# This file should ensure the existence of records required to run the application in every environment (production, +# development, test). The code here should be idempotent so that it can be executed at any point in every environment. +# The data can then be loaded with the bin/rails db:seed command (or created alongside the database with db:setup). +# +# Example: +# +# ["Action", "Comedy", "Drama", "Horror"].each do |genre_name| +# MovieGenre.find_or_create_by!(name: genre_name) +# end diff --git a/cable-bench-falcon/docker-compose.yml b/cable-bench-falcon/docker-compose.yml new file mode 100644 index 0000000..637d9b0 --- /dev/null +++ b/cable-bench-falcon/docker-compose.yml @@ -0,0 +1,21 @@ +# Local validation of the AsyncCable (Falcon) target against the production +# image. Mirrors the Railway topology (Falcon + Redis cable adapter) so we can +# shake out config before deploying. +# +# docker compose up --build +# +# AsyncCable: ws://localhost:3003/cable broadcast http://localhost:3003/_bench/broadcast +services: + redis: + image: redis:7-alpine + ports: ["6379:6379"] + + rails-asynccable: + build: . + image: cable-bench-falcon:local + environment: + PORT: "3000" + WEB_CONCURRENCY: "2" + REDIS_URL: redis://redis:6379/1 + ports: ["3003:3000"] + depends_on: [redis] diff --git a/cable-bench-falcon/lib/tasks/.keep b/cable-bench-falcon/lib/tasks/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench-falcon/public/400.html b/cable-bench-falcon/public/400.html new file mode 100644 index 0000000..640de03 --- /dev/null +++ b/cable-bench-falcon/public/400.html @@ -0,0 +1,135 @@ + + + + + + + The server cannot process the request due to a client error (400 Bad Request) + + + + + + + + + + + + + +
+
+ +
+
+

The server cannot process the request due to a client error. Please check the request and try again. If you're the application owner check the logs for more information.

+
+
+ + + + diff --git a/cable-bench-falcon/public/404.html b/cable-bench-falcon/public/404.html new file mode 100644 index 0000000..d7f0f14 --- /dev/null +++ b/cable-bench-falcon/public/404.html @@ -0,0 +1,135 @@ + + + + + + + The page you were looking for doesn't exist (404 Not found) + + + + + + + + + + + + + +
+
+ +
+
+

The page you were looking for doesn't exist. You may have mistyped the address or the page may have moved. If you're the application owner check the logs for more information.

+
+
+ + + + diff --git a/cable-bench-falcon/public/406-unsupported-browser.html b/cable-bench-falcon/public/406-unsupported-browser.html new file mode 100644 index 0000000..43d2811 --- /dev/null +++ b/cable-bench-falcon/public/406-unsupported-browser.html @@ -0,0 +1,135 @@ + + + + + + + Your browser is not supported (406 Not Acceptable) + + + + + + + + + + + + + +
+
+ +
+
+

Your browser is not supported.
Please upgrade your browser to continue.

+
+
+ + + + diff --git a/cable-bench-falcon/public/422.html b/cable-bench-falcon/public/422.html new file mode 100644 index 0000000..f12fb4a --- /dev/null +++ b/cable-bench-falcon/public/422.html @@ -0,0 +1,135 @@ + + + + + + + The change you wanted was rejected (422 Unprocessable Entity) + + + + + + + + + + + + + +
+
+ +
+
+

The change you wanted was rejected. Maybe you tried to change something you didn't have access to. If you're the application owner check the logs for more information.

+
+
+ + + + diff --git a/cable-bench-falcon/public/500.html b/cable-bench-falcon/public/500.html new file mode 100644 index 0000000..e4eb18a --- /dev/null +++ b/cable-bench-falcon/public/500.html @@ -0,0 +1,135 @@ + + + + + + + We're sorry, but something went wrong (500 Internal Server Error) + + + + + + + + + + + + + +
+
+ +
+
+

We're sorry, but something went wrong.
If you're the application owner check the logs for more information.

+
+
+ + + + diff --git a/cable-bench-falcon/public/icon.png b/cable-bench-falcon/public/icon.png new file mode 100644 index 0000000..c4c9dbf Binary files /dev/null and b/cable-bench-falcon/public/icon.png differ diff --git a/cable-bench-falcon/public/icon.svg b/cable-bench-falcon/public/icon.svg new file mode 100644 index 0000000..04b34bf --- /dev/null +++ b/cable-bench-falcon/public/icon.svg @@ -0,0 +1,3 @@ + + + diff --git a/cable-bench-falcon/public/robots.txt b/cable-bench-falcon/public/robots.txt new file mode 100644 index 0000000..c19f78a --- /dev/null +++ b/cable-bench-falcon/public/robots.txt @@ -0,0 +1 @@ +# See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file diff --git a/cable-bench-falcon/railway.toml b/cable-bench-falcon/railway.toml new file mode 100644 index 0000000..134d13f --- /dev/null +++ b/cable-bench-falcon/railway.toml @@ -0,0 +1,11 @@ +# Per-service Railway config for the Rails cable-bench targets +# (rails-solidcable / rails-actioncable / rails-anycable). +# +# The repo-root railway.toml points at backend/Dockerfile, which is wrong for +# this service. Uploading this file as the build root via +# `railway up --service --path-as-root cable-bench/` overrides the repo +# default and uses this local Dockerfile. The three services share the image +# and differ only by the BENCH_MODE env var. + +[build] +dockerfilePath = "Dockerfile" diff --git a/cable-bench-falcon/script/.keep b/cable-bench-falcon/script/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench-falcon/storage/.keep b/cable-bench-falcon/storage/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench-falcon/vendor/.keep b/cable-bench-falcon/vendor/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench-falcon/vendor/javascript/.keep b/cable-bench-falcon/vendor/javascript/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench/.dockerignore b/cable-bench/.dockerignore new file mode 100644 index 0000000..6751f09 --- /dev/null +++ b/cable-bench/.dockerignore @@ -0,0 +1,15 @@ +.git +.gitignore +# No encrypted credentials are used (SECRET_KEY_BASE_DUMMY=1 supplies the key). +# Copying master.key as a root-owned 0600 file breaks boot under USER rails. +/config/master.key +/log/* +/tmp/* +/storage/* +!/storage/.keep +/node_modules +/.bundle +/vendor/bundle +*.sqlite3 +*.sqlite3-* +.DS_Store diff --git a/cable-bench/.gitattributes b/cable-bench/.gitattributes new file mode 100644 index 0000000..8dc4323 --- /dev/null +++ b/cable-bench/.gitattributes @@ -0,0 +1,9 @@ +# See https://git-scm.com/docs/gitattributes for more about git attribute files. + +# Mark the database schema as having been generated. +db/schema.rb linguist-generated + +# Mark any vendored files as having been vendored. +vendor/* linguist-vendored +config/credentials/*.yml.enc diff=rails_credentials +config/credentials.yml.enc diff=rails_credentials diff --git a/cable-bench/.gitignore b/cable-bench/.gitignore new file mode 100644 index 0000000..fbcab40 --- /dev/null +++ b/cable-bench/.gitignore @@ -0,0 +1,35 @@ +# See https://help.github.com/articles/ignoring-files for more about ignoring files. +# +# Temporary files generated by your text editor or operating system +# belong in git's global ignore instead: +# `$XDG_CONFIG_HOME/git/ignore` or `~/.config/git/ignore` + +# Ignore bundler config. +/.bundle + +# Ignore all environment files. +/.env* + +# Ignore all logfiles and tempfiles. +/log/* +/tmp/* +!/log/.keep +!/tmp/.keep + +# Ignore pidfiles, but keep the directory. +/tmp/pids/* +!/tmp/pids/ +!/tmp/pids/.keep + +# Ignore storage (uploaded files in development and any SQLite databases). +/storage/* +!/storage/.keep +/tmp/storage/* +!/tmp/storage/ +!/tmp/storage/.keep + +/public/assets + +# Ignore key files for decrypting credentials and more. +/config/*.key + diff --git a/cable-bench/.rubocop.yml b/cable-bench/.rubocop.yml new file mode 100644 index 0000000..f9d86d4 --- /dev/null +++ b/cable-bench/.rubocop.yml @@ -0,0 +1,8 @@ +# Omakase Ruby styling for Rails +inherit_gem: { rubocop-rails-omakase: rubocop.yml } + +# Overwrite or add rules to create your own house style +# +# # Use `[a, [b, c]]` not `[ a, [ b, c ] ]` +# Layout/SpaceInsideArrayLiteralBrackets: +# Enabled: false diff --git a/cable-bench/.ruby-version b/cable-bench/.ruby-version new file mode 100644 index 0000000..f989260 --- /dev/null +++ b/cable-bench/.ruby-version @@ -0,0 +1 @@ +3.4.4 diff --git a/cable-bench/Dockerfile b/cable-bench/Dockerfile new file mode 100644 index 0000000..c4ee015 --- /dev/null +++ b/cable-bench/Dockerfile @@ -0,0 +1,36 @@ +# syntax=docker/dockerfile:1 +# Single image for the three Rails cable benchmark modes; bin/bench-entrypoint +# picks the adapter and process from BENCH_MODE. Mirrors the anycable-pro / +# socketioxide per-service pattern (railway.toml + --path-as-root cable-bench/). +ARG RUBY_VERSION=3.4.4 +FROM ruby:$RUBY_VERSION-slim AS base +WORKDIR /app +ENV RAILS_ENV=production \ + BUNDLE_DEPLOYMENT=1 \ + BUNDLE_WITHOUT=development:test \ + BUNDLE_PATH=/usr/local/bundle \ + RAILS_LOG_TO_STDOUT=1 \ + RAILS_SERVE_STATIC_FILES=1 \ + SECRET_KEY_BASE_DUMMY=1 +RUN apt-get update -qq && \ + apt-get install --no-install-recommends -y curl && \ + rm -rf /var/lib/apt/lists/* + +FROM base AS build +RUN apt-get update -qq && \ + apt-get install --no-install-recommends -y build-essential git pkg-config libyaml-dev && \ + rm -rf /var/lib/apt/lists/* +COPY Gemfile Gemfile.lock ./ +RUN bundle install && rm -rf "${BUNDLE_PATH}"/ruby/*/cache +COPY . . +RUN ./bin/rails assets:precompile + +FROM base +COPY --from=build /usr/local/bundle /usr/local/bundle +COPY --from=build /app /app +RUN useradd rails --create-home --shell /bin/bash && \ + mkdir -p storage tmp/pids log && \ + chown -R rails:rails db log storage tmp +USER rails +EXPOSE 3000 50051 +ENTRYPOINT ["./bin/bench-entrypoint"] diff --git a/cable-bench/Gemfile b/cable-bench/Gemfile new file mode 100644 index 0000000..ba9a907 --- /dev/null +++ b/cable-bench/Gemfile @@ -0,0 +1,58 @@ +source "https://rubygems.org" + +# Bundle edge Rails instead: gem "rails", github: "rails/rails", branch: "main" +gem "rails", "~> 8.1.3" +# The modern asset pipeline for Rails [https://github.com/rails/propshaft] +gem "propshaft" +# Use sqlite3 as the database for Active Record +gem "sqlite3", ">= 2.1" +# Use the Puma web server [https://github.com/puma/puma] +gem "puma", ">= 5.0" +# Use JavaScript with ESM import maps [https://github.com/rails/importmap-rails] +gem "importmap-rails" +# Hotwire's SPA-like page accelerator [https://turbo.hotwired.dev] +gem "turbo-rails" +# Hotwire's modest JavaScript framework [https://stimulus.hotwired.dev] +gem "stimulus-rails" + +# Use Active Model has_secure_password [https://guides.rubyonrails.org/active_model_basics.html#securepassword] +# gem "bcrypt", "~> 3.1.7" + +# Windows does not include zoneinfo files, so bundle the tzinfo-data gem +gem "tzinfo-data", platforms: %i[ windows jruby ] + +# Use the database-backed adapters for Rails.cache, Active Job, and Action Cable +gem "solid_cache" +gem "solid_queue" +gem "solid_cable" + +# Benchmark targets: classic Action Cable Redis adapter + AnyCable (Go gateway +# over gRPC with the extended Action Cable protocol). Adapter is chosen at boot +# by CABLE_ADAPTER (see config/cable.yml). +gem "redis", ">= 4.0" +gem "anycable-rails", "~> 1.6" + +# Reduces boot times through caching; required in config/boot.rb +gem "bootsnap", require: false + +# Add HTTP asset caching/compression and X-Sendfile acceleration to Puma [https://github.com/basecamp/thruster/] +gem "thruster", require: false + +group :development, :test do + # See https://guides.rubyonrails.org/debugging_rails_applications.html#debugging-with-the-debug-gem + gem "debug", platforms: %i[ mri windows ], require: "debug/prelude" + + # Audits gems for known security defects (use config/bundler-audit.yml to ignore issues) + gem "bundler-audit", require: false + + # Static analysis for security vulnerabilities [https://brakemanscanner.org/] + gem "brakeman", require: false + + # Omakase Ruby styling [https://github.com/rails/rubocop-rails-omakase/] + gem "rubocop-rails-omakase", require: false +end + +group :development do + # Use console on exceptions pages [https://github.com/rails/web-console] + gem "web-console" +end diff --git a/cable-bench/Gemfile.lock b/cable-bench/Gemfile.lock new file mode 100644 index 0000000..13e829c --- /dev/null +++ b/cable-bench/Gemfile.lock @@ -0,0 +1,560 @@ +GEM + remote: https://rubygems.org/ + specs: + action_text-trix (2.1.19) + railties + actioncable (8.1.3) + actionpack (= 8.1.3) + activesupport (= 8.1.3) + nio4r (~> 2.0) + websocket-driver (>= 0.6.1) + zeitwerk (~> 2.6) + actionmailbox (8.1.3) + actionpack (= 8.1.3) + activejob (= 8.1.3) + activerecord (= 8.1.3) + activestorage (= 8.1.3) + activesupport (= 8.1.3) + mail (>= 2.8.0) + actionmailer (8.1.3) + actionpack (= 8.1.3) + actionview (= 8.1.3) + activejob (= 8.1.3) + activesupport (= 8.1.3) + mail (>= 2.8.0) + rails-dom-testing (~> 2.2) + actionpack (8.1.3) + actionview (= 8.1.3) + activesupport (= 8.1.3) + nokogiri (>= 1.8.5) + rack (>= 2.2.4) + rack-session (>= 1.0.1) + rack-test (>= 0.6.3) + rails-dom-testing (~> 2.2) + rails-html-sanitizer (~> 1.6) + useragent (~> 0.16) + actiontext (8.1.3) + action_text-trix (~> 2.1.15) + actionpack (= 8.1.3) + activerecord (= 8.1.3) + activestorage (= 8.1.3) + activesupport (= 8.1.3) + globalid (>= 0.6.0) + nokogiri (>= 1.8.5) + actionview (8.1.3) + activesupport (= 8.1.3) + builder (~> 3.1) + erubi (~> 1.11) + rails-dom-testing (~> 2.2) + rails-html-sanitizer (~> 1.6) + activejob (8.1.3) + activesupport (= 8.1.3) + globalid (>= 0.3.6) + activemodel (8.1.3) + activesupport (= 8.1.3) + activerecord (8.1.3) + activemodel (= 8.1.3) + activesupport (= 8.1.3) + timeout (>= 0.4.0) + activestorage (8.1.3) + actionpack (= 8.1.3) + activejob (= 8.1.3) + activerecord (= 8.1.3) + activesupport (= 8.1.3) + marcel (~> 1.0) + activesupport (8.1.3) + base64 + bigdecimal + concurrent-ruby (~> 1.0, >= 1.3.1) + connection_pool (>= 2.2.5) + drb + i18n (>= 1.6, < 2) + json + logger (>= 1.4.2) + minitest (>= 5.1) + securerandom (>= 0.3) + tzinfo (~> 2.0, >= 2.0.5) + uri (>= 0.13.1) + anycable (1.6.4) + anycable-core (= 1.6.4) + grpc (~> 1.6) + anycable-core (1.6.4) + anyway_config (~> 2.2) + base64 (>= 0.2) + google-protobuf (~> 4) + stringio (~> 3) + anycable-rails (1.6.2) + anycable + anycable-rails-core (= 1.6.2) + anycable-rails-core (1.6.2) + actioncable (>= 7.0, < 9.0) + anycable-core (~> 1.6.0) + globalid + anyway_config (2.8.0) + ruby-next-core (~> 1.0) + ast (2.4.3) + base64 (0.3.0) + bigdecimal (4.1.2) + bindex (0.8.1) + bootsnap (1.24.6) + msgpack (~> 1.2) + brakeman (8.0.5) + racc + builder (3.3.0) + bundler-audit (0.9.3) + bundler (>= 1.2.0) + thor (~> 1.0) + concurrent-ruby (1.3.7) + connection_pool (3.0.2) + crass (1.0.6) + date (3.5.1) + debug (1.11.1) + irb (~> 1.10) + reline (>= 0.3.8) + drb (2.2.3) + erb (6.0.4) + erubi (1.13.1) + et-orbi (1.4.0) + tzinfo + fugit (1.12.2) + et-orbi (~> 1.4) + raabro (~> 1.4) + globalid (1.4.0) + activesupport (>= 6.1) + google-protobuf (4.35.1) + bigdecimal + rake (~> 13.3) + google-protobuf (4.35.1-aarch64-linux-gnu) + bigdecimal + rake (~> 13.3) + google-protobuf (4.35.1-aarch64-linux-musl) + bigdecimal + rake (~> 13.3) + google-protobuf (4.35.1-arm64-darwin) + bigdecimal + rake (~> 13.3) + google-protobuf (4.35.1-x86_64-darwin) + bigdecimal + rake (~> 13.3) + google-protobuf (4.35.1-x86_64-linux-gnu) + bigdecimal + rake (~> 13.3) + google-protobuf (4.35.1-x86_64-linux-musl) + bigdecimal + rake (~> 13.3) + googleapis-common-protos-types (1.23.0) + google-protobuf (~> 4.26) + grpc (1.81.1) + google-protobuf (>= 3.25, < 5.0) + googleapis-common-protos-types (~> 1.0) + grpc (1.81.1-aarch64-linux-gnu) + google-protobuf (>= 3.25, < 5.0) + googleapis-common-protos-types (~> 1.0) + grpc (1.81.1-aarch64-linux-musl) + google-protobuf (>= 3.25, < 5.0) + googleapis-common-protos-types (~> 1.0) + grpc (1.81.1-arm64-darwin) + google-protobuf (>= 3.25, < 5.0) + googleapis-common-protos-types (~> 1.0) + grpc (1.81.1-x86_64-darwin) + google-protobuf (>= 3.25, < 5.0) + googleapis-common-protos-types (~> 1.0) + grpc (1.81.1-x86_64-linux-gnu) + google-protobuf (>= 3.25, < 5.0) + googleapis-common-protos-types (~> 1.0) + grpc (1.81.1-x86_64-linux-musl) + google-protobuf (>= 3.25, < 5.0) + googleapis-common-protos-types (~> 1.0) + i18n (1.15.2) + concurrent-ruby (~> 1.0) + importmap-rails (2.2.3) + actionpack (>= 6.0.0) + activesupport (>= 6.0.0) + railties (>= 6.0.0) + io-console (0.8.2) + irb (1.18.0) + pp (>= 0.6.0) + prism (>= 1.3.0) + rdoc (>= 4.0.0) + reline (>= 0.4.2) + json (2.20.0) + language_server-protocol (3.17.0.5) + lint_roller (1.1.0) + logger (1.7.0) + loofah (2.25.1) + crass (~> 1.0.2) + nokogiri (>= 1.12.0) + mail (2.9.0) + logger + mini_mime (>= 0.1.1) + net-imap + net-pop + net-smtp + marcel (1.2.1) + mini_mime (1.1.5) + minitest (6.0.6) + drb (~> 2.0) + prism (~> 1.5) + msgpack (1.8.3) + net-imap (0.6.4.1) + date + net-protocol + net-pop (0.1.2) + net-protocol + net-protocol (0.2.2) + timeout + net-smtp (0.5.1) + net-protocol + nio4r (2.7.5) + nokogiri (1.19.4-aarch64-linux-gnu) + racc (~> 1.4) + nokogiri (1.19.4-aarch64-linux-musl) + racc (~> 1.4) + nokogiri (1.19.4-arm-linux-gnu) + racc (~> 1.4) + nokogiri (1.19.4-arm-linux-musl) + racc (~> 1.4) + nokogiri (1.19.4-arm64-darwin) + racc (~> 1.4) + nokogiri (1.19.4-x86_64-darwin) + racc (~> 1.4) + nokogiri (1.19.4-x86_64-linux-gnu) + racc (~> 1.4) + nokogiri (1.19.4-x86_64-linux-musl) + racc (~> 1.4) + parallel (2.1.0) + parser (3.3.11.1) + ast (~> 2.4.1) + racc + pp (0.6.4) + prettyprint + prettyprint (0.2.0) + prism (1.9.0) + propshaft (1.3.2) + actionpack (>= 7.0.0) + activesupport (>= 7.0.0) + rack + psych (5.4.0) + date + stringio + puma (8.0.2) + nio4r (~> 2.0) + raabro (1.4.0) + racc (1.8.1) + rack (3.2.6) + rack-session (2.1.2) + base64 (>= 0.1.0) + rack (>= 3.0.0) + rack-test (2.2.0) + rack (>= 1.3) + rackup (2.3.1) + rack (>= 3) + rails (8.1.3) + actioncable (= 8.1.3) + actionmailbox (= 8.1.3) + actionmailer (= 8.1.3) + actionpack (= 8.1.3) + actiontext (= 8.1.3) + actionview (= 8.1.3) + activejob (= 8.1.3) + activemodel (= 8.1.3) + activerecord (= 8.1.3) + activestorage (= 8.1.3) + activesupport (= 8.1.3) + bundler (>= 1.15.0) + railties (= 8.1.3) + rails-dom-testing (2.3.0) + activesupport (>= 5.0.0) + minitest + nokogiri (>= 1.6) + rails-html-sanitizer (1.7.0) + loofah (~> 2.25) + nokogiri (>= 1.15.7, != 1.16.7, != 1.16.6, != 1.16.5, != 1.16.4, != 1.16.3, != 1.16.2, != 1.16.1, != 1.16.0.rc1, != 1.16.0) + railties (8.1.3) + actionpack (= 8.1.3) + activesupport (= 8.1.3) + irb (~> 1.13) + rackup (>= 1.0.0) + rake (>= 12.2) + thor (~> 1.0, >= 1.2.2) + tsort (>= 0.2) + zeitwerk (~> 2.6) + rainbow (3.1.1) + rake (13.4.2) + rdoc (7.2.0) + erb + psych (>= 4.0.0) + tsort + redis (5.4.1) + redis-client (>= 0.22.0) + redis-client (0.30.0) + connection_pool + regexp_parser (2.12.0) + reline (0.6.3) + io-console (~> 0.5) + rubocop (1.88.0) + json (~> 2.3) + language_server-protocol (~> 3.17.0.2) + lint_roller (~> 1.1.0) + parallel (>= 1.10) + parser (>= 3.3.0.2) + rainbow (>= 2.2.2, < 4.0) + regexp_parser (>= 2.9.3, < 3.0) + rubocop-ast (>= 1.49.0, < 2.0) + ruby-progressbar (~> 1.7) + unicode-display_width (>= 2.4.0, < 4.0) + rubocop-ast (1.49.1) + parser (>= 3.3.7.2) + prism (~> 1.7) + rubocop-performance (1.26.1) + lint_roller (~> 1.1) + rubocop (>= 1.75.0, < 2.0) + rubocop-ast (>= 1.47.1, < 2.0) + rubocop-rails (2.35.5) + activesupport (>= 4.2.0) + lint_roller (~> 1.1) + rack (>= 1.1) + rubocop (>= 1.75.0, < 2.0) + rubocop-ast (>= 1.44.0, < 2.0) + rubocop-rails-omakase (1.1.0) + rubocop (>= 1.72) + rubocop-performance (>= 1.24) + rubocop-rails (>= 2.30) + ruby-next-core (1.2.0) + ruby-progressbar (1.13.0) + securerandom (0.4.1) + solid_cable (4.0.0) + actioncable (>= 7.2) + activejob (>= 7.2) + activerecord (>= 7.2) + railties (>= 7.2) + solid_cache (1.0.10) + activejob (>= 7.2) + activerecord (>= 7.2) + railties (>= 7.2) + solid_queue (1.4.0) + activejob (>= 7.1) + activerecord (>= 7.1) + concurrent-ruby (>= 1.3.1) + fugit (~> 1.11) + railties (>= 7.1) + thor (>= 1.3.1) + sqlite3 (2.9.5-aarch64-linux-gnu) + sqlite3 (2.9.5-aarch64-linux-musl) + sqlite3 (2.9.5-arm-linux-gnu) + sqlite3 (2.9.5-arm-linux-musl) + sqlite3 (2.9.5-arm64-darwin) + sqlite3 (2.9.5-x86_64-darwin) + sqlite3 (2.9.5-x86_64-linux-gnu) + sqlite3 (2.9.5-x86_64-linux-musl) + stimulus-rails (1.3.4) + railties (>= 6.0.0) + stringio (3.2.0) + thor (1.5.0) + thruster (0.1.21) + thruster (0.1.21-aarch64-linux) + thruster (0.1.21-arm64-darwin) + thruster (0.1.21-x86_64-darwin) + thruster (0.1.21-x86_64-linux) + timeout (0.6.1) + tsort (0.2.0) + turbo-rails (2.0.23) + actionpack (>= 7.1.0) + railties (>= 7.1.0) + tzinfo (2.0.6) + concurrent-ruby (~> 1.0) + unicode-display_width (3.2.0) + unicode-emoji (~> 4.1) + unicode-emoji (4.2.0) + uri (1.1.1) + useragent (0.16.11) + web-console (4.3.0) + actionview (>= 8.0.0) + bindex (>= 0.4.0) + railties (>= 8.0.0) + websocket-driver (0.8.2) + base64 + websocket-extensions (>= 0.1.0) + websocket-extensions (0.1.5) + zeitwerk (2.8.2) + +PLATFORMS + aarch64-linux + aarch64-linux-gnu + aarch64-linux-musl + arm-linux-gnu + arm-linux-musl + arm64-darwin + x86_64-darwin + x86_64-linux + x86_64-linux-gnu + x86_64-linux-musl + +DEPENDENCIES + anycable-rails (~> 1.6) + bootsnap + brakeman + bundler-audit + debug + importmap-rails + propshaft + puma (>= 5.0) + rails (~> 8.1.3) + redis (>= 4.0) + rubocop-rails-omakase + solid_cable + solid_cache + solid_queue + sqlite3 (>= 2.1) + stimulus-rails + thruster + turbo-rails + tzinfo-data + web-console + +CHECKSUMS + action_text-trix (2.1.19) sha256=7012f59421009cf284aa651294896414d653a61a2417c9b8714c8476d2f74009 + actioncable (8.1.3) sha256=e5bc7f75e44e6a22de29c4f43176927c3a9ce4824464b74ed18d8226e75a80f0 + actionmailbox (8.1.3) sha256=df7da474eaa0e70df4ed5a6fef66eb3b3b0f2dbf7f14518deee8d77f1b4aae59 + actionmailer (8.1.3) sha256=831f724891bb70d0aaa4d76581a6321124b6a752cb655c9346aae5479318448d + actionpack (8.1.3) sha256=af998cae4d47c5d581a2cc363b5c77eb718b7c4b45748d81b1887b25621c29a3 + actiontext (8.1.3) sha256=d291019c00e1ea9e6463011fa214f6081a56d7b9a1d224e7d3f6384c1dafc7d2 + actionview (8.1.3) sha256=1347c88c7f3edb38100c5ce0e9fb5e62d7755f3edc1b61cce2eb0b2c6ea2fd5d + activejob (8.1.3) sha256=a149b1766aa8204c3c3da7309e4becd40fcd5529c348cffbf6c9b16b565fe8d3 + activemodel (8.1.3) sha256=90c05cbe4cef3649b8f79f13016191ea94c4525ce4a5c0fb7ef909c4b91c8219 + activerecord (8.1.3) sha256=8003be7b2466ba0a2a670e603eeb0a61dd66058fccecfc49901e775260ac70ab + activestorage (8.1.3) sha256=0564ce9309143951a67615e1bb4e090ee54b8befed417133cae614479b46384d + activesupport (8.1.3) sha256=21a5e0dfbd4c3ddd9e1317ec6a4d782fa226e7867dc70b0743acda81a1dca20e + anycable (1.6.4) sha256=a440dd44d4d8dc45b18751b041e775e0b98df458b1d8043b511903423a2f9118 + anycable-core (1.6.4) sha256=17073a3c744d7057e43bc7ed2cbdc1d50f173bb0c874109c55a49c7b95819396 + anycable-rails (1.6.2) sha256=954fd30f0f91825a06122da85abbf5b962d22fe0f4f3a2b5b262d4b98f5f3582 + anycable-rails-core (1.6.2) sha256=868e31c1df0aa3096073fa59df5c9815cdaaf2e798578d874f598044418f6d13 + anyway_config (2.8.0) sha256=f6797a7231f81202dcd3d0c07284e836e45713e761d320180348b13a5c7c9306 + ast (2.4.3) sha256=954615157c1d6a382bc27d690d973195e79db7f55e9765ac7c481c60bdb4d383 + base64 (0.3.0) sha256=27337aeabad6ffae05c265c450490628ef3ebd4b67be58257393227588f5a97b + bigdecimal (4.1.2) sha256=53d217666027eab4280346fba98e7d5b66baaae1b9c3c1c0ffe89d48188a3fbd + bindex (0.8.1) sha256=7b1ecc9dc539ed8bccfc8cb4d2732046227b09d6f37582ff12e50a5047ceb17e + bootsnap (1.24.6) sha256=c60bab88c70332290f0a2636a288f675299eb4f804a02a3c085b42eca9da164a + brakeman (8.0.5) sha256=03735f9690d3fd4b32d66aacbf0a6d15a84266bdd06b32c05c8ecc8f6021d2be + builder (3.3.0) sha256=497918d2f9dca528fdca4b88d84e4ef4387256d984b8154e9d5d3fe5a9c8835f + bundler-audit (0.9.3) sha256=81c8766c71e47d0d28a0f98c7eed028539f21a6ea3cd8f685eb6f42333c9b4e9 + concurrent-ruby (1.3.7) sha256=4412caec3a5ea2e5fdc52076724c071a81f2c0593d83b2ac8cbb8ca63b3151b0 + connection_pool (3.0.2) sha256=33fff5ba71a12d2aa26cb72b1db8bba2a1a01823559fb01d29eb74c286e62e0a + crass (1.0.6) sha256=dc516022a56e7b3b156099abc81b6d2b08ea1ed12676ac7a5657617f012bd45d + date (3.5.1) sha256=750d06384d7b9c15d562c76291407d89e368dda4d4fff957eb94962d325a0dc0 + debug (1.11.1) sha256=2e0b0ac6119f2207a6f8ac7d4a73ca8eb4e440f64da0a3136c30343146e952b6 + drb (2.2.3) sha256=0b00d6fdb50995fe4a45dea13663493c841112e4068656854646f418fda13373 + erb (6.0.4) sha256=38e3803694be357fe2bfe312487c74beaf9fb4e5beb3e22498952fe1645b95d9 + erubi (1.13.1) sha256=a082103b0885dbc5ecf1172fede897f9ebdb745a4b97a5e8dc63953db1ee4ad9 + et-orbi (1.4.0) sha256=6c7e3c90779821f9e3b324c5e96fda9767f72995d6ae435b96678a4f3e2de8bc + fugit (1.12.2) sha256=643f2bf28db263bd400cbf8e0dd8b76b2c9b94bdb130e12d2394de04d9c20e5e + globalid (1.4.0) sha256=037f12fbf1d9d7a014d501c2d5c77356fd4ddd96d7a7991d6700bba96706f427 + google-protobuf (4.35.1) sha256=a3a6471331d918f58dfa4d014a8f6286f0af2cf4840216bde52fcf2ea3fe3726 + google-protobuf (4.35.1-aarch64-linux-gnu) sha256=50ca44d0eeff3f8475e630a1accdd974256f3510694d574e2c9d6119ea8bc9e1 + google-protobuf (4.35.1-aarch64-linux-musl) sha256=d5c65cef6bd6498a9e5ed5f88cf6cf7e341c10b0a005e32137d5d1a2b6e8c18a + google-protobuf (4.35.1-arm64-darwin) sha256=d9c957df04fa89c749fa9a72a7b383eb4296efc9b2303dc6fd6fbe39c698ad6b + google-protobuf (4.35.1-x86_64-darwin) sha256=66b62b4df00931018a692806df66393efa960d6d2b7da69735187249f950d3ee + google-protobuf (4.35.1-x86_64-linux-gnu) sha256=c786439087512a3fbd199e9897d265b855f951d4027e218ea55e858d45969edd + google-protobuf (4.35.1-x86_64-linux-musl) sha256=91890eb0002934a339fdb7d77a147c46b7474b6799db27872b747b905837f744 + googleapis-common-protos-types (1.23.0) sha256=992e740a523794d9fc5f29a504465d8fc737aaa16c930fe7228e3346860faf0a + grpc (1.81.1) sha256=9e5772153fe13a389654e9d397a90400f0077307fb4369f79f941d96c72e89cb + grpc (1.81.1-aarch64-linux-gnu) sha256=aa24d7253a2b15d2c3570265a2f58bb8aca0ecd2e1116b7a259d633ebf582445 + grpc (1.81.1-aarch64-linux-musl) sha256=36af13b712e33b323878096ddcab7ec75d107d2e03ffce4aa2fcb70835747c8c + grpc (1.81.1-arm64-darwin) sha256=6def610da088597e1a94bafbb36a92f19c456cb7df7bbe9618aeb63e8805ae9a + grpc (1.81.1-x86_64-darwin) sha256=f8c44cdfc26270247f2beb49ecbc983d0c184b6c24ac4670d1d24f0efba811ed + grpc (1.81.1-x86_64-linux-gnu) sha256=cf1053a594d026d2ec560457111258cd207bc67847aab24f96e04e2fbeddc4dd + grpc (1.81.1-x86_64-linux-musl) sha256=c4e2d53566dedfb7e76b2226c18f92f6378b29fefb80879b2dabe17480faa555 + i18n (1.15.2) sha256=00f9eb62412fe593b2a65a97daa75300d37abb8f7202ec748e94b6d46a9dd1b5 + importmap-rails (2.2.3) sha256=7101be2a4dc97cf1558fb8f573a718404c5f6bcfe94f304bf1f39e444feeb16a + io-console (0.8.2) sha256=d6e3ae7a7cc7574f4b8893b4fca2162e57a825b223a177b7afa236c5ef9814cc + irb (1.18.0) sha256=de9454a0703a54704b9811a5ef31a60c86949fbf4013fcf244fabc7c775248e3 + json (2.20.0) sha256=9362bc6e55a952b056abf9167cf053358181c904cb70cd6eee0808ea830fc32b + language_server-protocol (3.17.0.5) sha256=fd1e39a51a28bf3eec959379985a72e296e9f9acfce46f6a79d31ca8760803cc + lint_roller (1.1.0) sha256=2c0c845b632a7d172cb849cc90c1bce937a28c5c8ccccb50dfd46a485003cc87 + logger (1.7.0) sha256=196edec7cc44b66cfb40f9755ce11b392f21f7967696af15d274dde7edff0203 + loofah (2.25.1) sha256=d436c73dbd0c1147b16c4a41db097942d217303e1f7728704b37e4df9f6d2e04 + mail (2.9.0) sha256=6fa6673ecd71c60c2d996260f9ee3dd387d4673b8169b502134659ece6d34941 + marcel (1.2.1) sha256=1678e9360e32f9eafa917c80029e2f6d10b2715c66a4b87b6d0da9b9cd1f859f + mini_mime (1.1.5) sha256=8681b7e2e4215f2a159f9400b5816d85e9d8c6c6b491e96a12797e798f8bccef + minitest (6.0.6) sha256=153ea36d1d987a62942382b61075745042a2b3123b1cd48f4c3675af9cc7d6f1 + msgpack (1.8.3) sha256=8bda4a6428d3244e50d6bd55854d354edbada88a4e1f4f5731a39a0f86bee6a1 + net-imap (0.6.4.1) sha256=29f0360d75a7efd3539f16ac1957dea5c0a51ddeceb348db4553c3120914ea0d + net-pop (0.1.2) sha256=848b4e982013c15b2f0382792268763b748cce91c9e91e36b0f27ed26420dff3 + net-protocol (0.2.2) sha256=aa73e0cba6a125369de9837b8d8ef82a61849360eba0521900e2c3713aa162a8 + net-smtp (0.5.1) sha256=ed96a0af63c524fceb4b29b0d352195c30d82dd916a42f03c62a3a70e5b70736 + nio4r (2.7.5) sha256=6c90168e48fb5f8e768419c93abb94ba2b892a1d0602cb06eef16d8b7df1dca1 + nokogiri (1.19.4-aarch64-linux-gnu) sha256=1269fb644a6de405057a53dd5c762b1209b43ca7424f839454d3dbc677c31a8f + nokogiri (1.19.4-aarch64-linux-musl) sha256=35c65b9ce72b3bb03207bdbe7067915019dc18c1b9b59139684bd6690fdd01af + nokogiri (1.19.4-arm-linux-gnu) sha256=a301313e38bb065d68239e79734bcd6f56fb6efaacebde29e9abf2a4735340ca + nokogiri (1.19.4-arm-linux-musl) sha256=588923c101bcfa78869734d247d25b598674323e7f22474fc468f6e5647311eb + nokogiri (1.19.4-arm64-darwin) sha256=a46db9853286e6597b36ebc6953817d15acf3a299583eb3f89fdc6f91dd63527 + nokogiri (1.19.4-x86_64-darwin) sha256=7fd17057d3e1f00e9954a74b3cd76595d3d4a5ef233b7ed9599047c204f70551 + nokogiri (1.19.4-x86_64-linux-gnu) sha256=379fae440b28915e3f19d752ce2dcf8465ed2b2fbefd2a7ca0dd497bc981a06a + nokogiri (1.19.4-x86_64-linux-musl) sha256=17dfb7c1fa194ae02fbf7c51a7afc8d278045ab3fdacfd86f91d02d7b274470b + parallel (2.1.0) sha256=b35258865c2e31134c5ecb708beaaf6772adf9d5efae28e93e99260877b09356 + parser (3.3.11.1) sha256=d17ace7aabe3e72c3cc94043714be27cc6f852f104d81aa284c2281aecc65d54 + pp (0.6.4) sha256=dfcb0fce700c41456265922884f9fe195d7fbb0674a3578e6c0f69588e82b570 + prettyprint (0.2.0) sha256=2bc9e15581a94742064a3cc8b0fb9d45aae3d03a1baa6ef80922627a0766f193 + prism (1.9.0) sha256=7b530c6a9f92c24300014919c9dcbc055bf4cdf51ec30aed099b06cd6674ef85 + propshaft (1.3.2) sha256=1d56a3e56a92c21bfc29caf07406b5386b00d4c47ddf357cf989a5a234b1389e + psych (5.4.0) sha256=14f72d69a611af663d7d70e4a7b67d9eb1f3ae9f8d916b478961d5a0075ba5b7 + puma (8.0.2) sha256=c8ed871dfbbe66448ea9ffd46692342d9804d4071522b52b5331b7b6e7b686fb + raabro (1.4.0) sha256=d4fa9ff5172391edb92b242eed8be802d1934b1464061ae5e70d80962c5da882 + racc (1.8.1) sha256=4a7f6929691dbec8b5209a0b373bc2614882b55fc5d2e447a21aaa691303d62f + rack (3.2.6) sha256=5ed78e1f73b2e25679bec7d45ee2d4483cc4146eb1be0264fc4d94cb5ef212c2 + rack-session (2.1.2) sha256=595434f8c0c3473ae7d7ac56ecda6cc6dfd9d37c0b2b5255330aa1576967ffe8 + rack-test (2.2.0) sha256=005a36692c306ac0b4a9350355ee080fd09ddef1148a5f8b2ac636c720f5c463 + rackup (2.3.1) sha256=6c79c26753778e90983761d677a48937ee3192b3ffef6bc963c0950f94688868 + rails (8.1.3) sha256=6d017ba5348c98fc909753a8169b21d44de14d2a0b92d140d1a966834c3c9cd3 + rails-dom-testing (2.3.0) sha256=8acc7953a7b911ca44588bf08737bc16719f431a1cc3091a292bca7317925c1d + rails-html-sanitizer (1.7.0) sha256=28b145cceaf9cc214a9874feaa183c3acba036c9592b19886e0e45efc62b1e89 + railties (8.1.3) sha256=913eb0e0cb520aac687ffd74916bd726d48fa21f47833c6292576ef6a286de22 + rainbow (3.1.1) sha256=039491aa3a89f42efa1d6dec2fc4e62ede96eb6acd95e52f1ad581182b79bc6a + rake (13.4.2) sha256=cb825b2bd5f1f8e91ca37bddb4b9aaf345551b4731da62949be002fa89283701 + rdoc (7.2.0) sha256=8650f76cd4009c3b54955eb5d7e3a075c60a57276766ebf36f9085e8c9f23192 + redis (5.4.1) sha256=b5e675b57ad22b15c9bcc765d5ac26f60b675408af916d31527af9bd5a81faae + redis-client (0.30.0) sha256=743f11ed42f0a41a0341554087b077479fec7e2d47a7c123fd90a12c0db5e477 + regexp_parser (2.12.0) sha256=35a916a1d63190ab5c9009457136ae5f3c0c7512d60291d0d1378ba18ce08ebb + reline (0.6.3) sha256=1198b04973565b36ec0f11542ab3f5cfeeec34823f4e54cebde90968092b1835 + rubocop (1.88.0) sha256=e420ddf1662d0ef34bc8a2910ac4b396a7ddda0b51a708264405241734b08e0b + rubocop-ast (1.49.1) sha256=4412f3ee70f6fe4546cc489548e0f6fcf76cafcfa80fa03af67098ffed755035 + rubocop-performance (1.26.1) sha256=cd19b936ff196df85829d264b522fd4f98b6c89ad271fa52744a8c11b8f71834 + rubocop-rails (2.35.5) sha256=f00b3c936002ba8e9ac62e8607c54bb24cda44b36e41b9c7e4f3872e1b0f3fe3 + rubocop-rails-omakase (1.1.0) sha256=2af73ac8ee5852de2919abbd2618af9c15c19b512c4cfc1f9a5d3b6ef009109d + ruby-next-core (1.2.0) sha256=f6a7d00bb5186cecbb02f7f1845a0f3a2c9788d35b6ccff5c9be3f0d46799b86 + ruby-progressbar (1.13.0) sha256=80fc9c47a9b640d6834e0dc7b3c94c9df37f08cb072b7761e4a71e22cff29b33 + securerandom (0.4.1) sha256=cc5193d414a4341b6e225f0cb4446aceca8e50d5e1888743fac16987638ea0b1 + solid_cable (4.0.0) sha256=8379680ef6bf36e195eb876a6306ea290f87d5fa10bc4a757bc2a918f83229b5 + solid_cache (1.0.10) sha256=bc05a2fb3ac78a6f43cbb5946679cf9db67dd30d22939ededc385cb93e120d41 + solid_queue (1.4.0) sha256=e6a18d196f0b27cb6e3c77c5b31258b05fb634f8ed64fb1866ed164047216c2a + sqlite3 (2.9.5-aarch64-linux-gnu) sha256=78075b6337d3d182c6d2b4691049ed45cd220826160c9ea18946bf6a1de200dc + sqlite3 (2.9.5-aarch64-linux-musl) sha256=18c801185deb4adc01ddb281e8f672a39e3d1729979ca91e39439cd3eac0402d + sqlite3 (2.9.5-arm-linux-gnu) sha256=1bdfca0c7d63998c60b0f4a8e3c8df2d33800ccc4abd2d612eddbbbc92a4c48b + sqlite3 (2.9.5-arm-linux-musl) sha256=bae1109d12b2e9f588455967729b008e1ff4feb7761749df695019c9079913c6 + sqlite3 (2.9.5-arm64-darwin) sha256=d0cf444a70fc9395d513cfbcc1e6719e224aa645314e3824cb0474c721425aa2 + sqlite3 (2.9.5-x86_64-darwin) sha256=8e9caae38bd7ebb29cbeee3e7ab1d12dc2327d9a1b92c7fcf0dda05589627a81 + sqlite3 (2.9.5-x86_64-linux-gnu) sha256=233dbcb6714148dd23bc5aeb33e8efd6eac974969564ddd5794c23d5f52b231e + sqlite3 (2.9.5-x86_64-linux-musl) sha256=e7d3a7474e8af0f96150c21abc203fbab5437206bfcdf11deab7741c0ca516f2 + stimulus-rails (1.3.4) sha256=765676ffa1f33af64ce026d26b48e8ffb2e0b94e0f50e9119e11d6107d67cb06 + stringio (3.2.0) sha256=c37cb2e58b4ffbd33fe5cd948c05934af997b36e0b6ca6fdf43afa234cf222e1 + thor (1.5.0) sha256=e3a9e55fe857e44859ce104a84675ab6e8cd59c650a49106a05f55f136425e73 + thruster (0.1.21) sha256=dc67928f36e5894844579a95e45637a5091db7a7ea05468ee8c2c6eb0a3f77cf + thruster (0.1.21-aarch64-linux) sha256=f5aff78fb7a6431ed3d6ab4bde03a89c461e9a73981dbc97d6990d85c3db235c + thruster (0.1.21-arm64-darwin) sha256=bd8db9f57fae2cbb3fe08ebab49cb47fe49608122dac23daf0ce709adfb9bfc8 + thruster (0.1.21-x86_64-darwin) sha256=ccd6acd144fad27856800edfa0573944018333fac8e10a2e5d09726b70c8b0db + thruster (0.1.21-x86_64-linux) sha256=6e2fbcf826540a72d3710ae4db072c2333287ac2ee57e7e52f35bc10900d74a7 + timeout (0.6.1) sha256=78f57368a7e7bbadec56971f78a3f5ecbcfb59b7fcbb0a3ed6ddc08a5094accb + tsort (0.2.0) sha256=9650a793f6859a43b6641671278f79cfead60ac714148aabe4e3f0060480089f + turbo-rails (2.0.23) sha256=ee0d90733aafff056cf51ff11e803d65e43cae258cc55f6492020ec1f9f9315f + tzinfo (2.0.6) sha256=8daf828cc77bcf7d63b0e3bdb6caa47e2272dcfaf4fbfe46f8c3a9df087a829b + unicode-display_width (3.2.0) sha256=0cdd96b5681a5949cdbc2c55e7b420facae74c4aaf9a9815eee1087cb1853c42 + unicode-emoji (4.2.0) sha256=519e69150f75652e40bf736106cfbc8f0f73aa3fb6a65afe62fefa7f80b0f80f + uri (1.1.1) sha256=379fa58d27ffb1387eaada68c749d1426738bd0f654d812fcc07e7568f5c57c6 + useragent (0.16.11) sha256=700e6413ad4bb954bb63547fa098dddf7b0ebe75b40cc6f93b8d54255b173844 + web-console (4.3.0) sha256=e13b71301cdfc2093f155b5aa3a622db80b4672d1f2f713119cc7ec7ac6a6da4 + websocket-driver (0.8.2) sha256=97c556b019bf3410b4961002ac501621e9322d3f8a7bc02161a09301cc4c4146 + websocket-extensions (0.1.5) sha256=1c6ba63092cda343eb53fc657110c71c754c56484aad42578495227d717a8241 + zeitwerk (2.8.2) sha256=7212a61311083c604184b1ea2574b9aa05cd14f855a0841c06985cabe9181d12 + +BUNDLED WITH + 4.0.4 diff --git a/cable-bench/README.md b/cable-bench/README.md new file mode 100644 index 0000000..7db80e4 --- /dev/null +++ b/cable-bench/README.md @@ -0,0 +1,24 @@ +# README + +This README would normally document whatever steps are necessary to get the +application up and running. + +Things you may want to cover: + +* Ruby version + +* System dependencies + +* Configuration + +* Database creation + +* Database initialization + +* How to run the test suite + +* Services (job queues, cache servers, search engines, etc.) + +* Deployment instructions + +* ... diff --git a/cable-bench/Rakefile b/cable-bench/Rakefile new file mode 100644 index 0000000..9a5ea73 --- /dev/null +++ b/cable-bench/Rakefile @@ -0,0 +1,6 @@ +# Add your own tasks in files placed in lib/tasks ending in .rake, +# for example lib/tasks/capistrano.rake, and they will automatically be available to Rake. + +require_relative "config/application" + +Rails.application.load_tasks diff --git a/cable-bench/app/assets/images/.keep b/cable-bench/app/assets/images/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench/app/assets/stylesheets/application.css b/cable-bench/app/assets/stylesheets/application.css new file mode 100644 index 0000000..fe93333 --- /dev/null +++ b/cable-bench/app/assets/stylesheets/application.css @@ -0,0 +1,10 @@ +/* + * This is a manifest file that'll be compiled into application.css. + * + * With Propshaft, assets are served efficiently without preprocessing steps. You can still include + * application-wide styles in this file, but keep in mind that CSS precedence will follow the standard + * cascading order, meaning styles declared later in the document or manifest will override earlier ones, + * depending on specificity. + * + * Consider organizing styles into separate files for maintainability. + */ diff --git a/cable-bench/app/channels/application_cable/channel.rb b/cable-bench/app/channels/application_cable/channel.rb new file mode 100644 index 0000000..d672697 --- /dev/null +++ b/cable-bench/app/channels/application_cable/channel.rb @@ -0,0 +1,4 @@ +module ApplicationCable + class Channel < ActionCable::Channel::Base + end +end diff --git a/cable-bench/app/channels/application_cable/connection.rb b/cable-bench/app/channels/application_cable/connection.rb new file mode 100644 index 0000000..333d36f --- /dev/null +++ b/cable-bench/app/channels/application_cable/connection.rb @@ -0,0 +1,7 @@ +module ApplicationCable + # The benchmark connects anonymous clients (the load generator opens raw + # WebSockets and subscribes to BenchmarkChannel). No app-level auth is + # performed, so every adapter is measured on the same handshake cost. + class Connection < ActionCable::Connection::Base + end +end diff --git a/cable-bench/app/channels/benchmark_channel.rb b/cable-bench/app/channels/benchmark_channel.rb new file mode 100644 index 0000000..c3b2ca2 --- /dev/null +++ b/cable-bench/app/channels/benchmark_channel.rb @@ -0,0 +1,16 @@ +# The one channel the harness subscribes to. A client subscribes with +# `{ channel: "BenchmarkChannel", stream_name: "" }` and receives every +# message broadcast to that stream. This is identical Action Cable code for +# all three adapters (Solid Cable, classic Action Cable / Redis, AnyCable) — +# what differs is the transport underneath, not the app. +class BenchmarkChannel < ApplicationCable::Channel + def subscribed + stream_from params[:stream_name] + end + + # Client-to-client fan-out used by the optional whispers test. Mirrors the + # Node socket.io server's whisper handler so the harness can reuse its driver. + def whisper(data) + ActionCable.server.broadcast(params[:stream_name], data) + end +end diff --git a/cable-bench/app/controllers/application_controller.rb b/cable-bench/app/controllers/application_controller.rb new file mode 100644 index 0000000..c353756 --- /dev/null +++ b/cable-bench/app/controllers/application_controller.rb @@ -0,0 +1,7 @@ +class ApplicationController < ActionController::Base + # Only allow modern browsers supporting webp images, web push, badges, import maps, CSS nesting, and CSS :has. + allow_browser versions: :modern + + # Changes to the importmap will invalidate the etag for HTML responses + stale_when_importmap_changes +end diff --git a/cable-bench/app/controllers/bench_controller.rb b/cable-bench/app/controllers/bench_controller.rb new file mode 100644 index 0000000..2ca8a13 --- /dev/null +++ b/cable-bench/app/controllers/bench_controller.rb @@ -0,0 +1,41 @@ +# The harness publishes through Rails so every adapter shares one publish +# path: POST /_bench/broadcast {stream, data} -> ActionCable.server.broadcast. +# For AnyCable this is patched to publish through the broker (extended +# protocol, reliable streams); for Action Cable / Solid Cable it goes through +# the Redis / DB adapter. Same code, different transport. +class BenchController < ActionController::Base + # The load generator posts JSON without a CSRF token. + skip_forgery_protection + + before_action :authorize, only: :broadcast + + # GET /health — liveness probe (open, no auth). + def health + render json: { status: "ok", mode: ENV.fetch("CABLE_ADAPTER", "solid_cable") } + end + + # POST /_bench/broadcast {stream, data} + # `data` arrives as a JSON string (the harness pre-serializes it); broadcast + # the parsed object so subscribers receive { seq, sentAt, text }. + def broadcast + payload = + begin + JSON.parse(params.require(:data)) + rescue JSON::ParserError, TypeError + params[:data] + end + ActionCable.server.broadcast(params.require(:stream), payload) + head :ok + end + + private + + # Optional bearer gate, matching the bench-runner's BENCH_RUNNER_TOKEN. If + # the env var is unset (local dev), the endpoint is open. + def authorize + expected = ENV["BENCH_RUNNER_TOKEN"] + return if expected.blank? + provided = request.headers["Authorization"].to_s.delete_prefix("Bearer ") + head :unauthorized unless ActiveSupport::SecurityUtils.secure_compare(provided, expected) + end +end diff --git a/cable-bench/app/controllers/concerns/.keep b/cable-bench/app/controllers/concerns/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench/app/helpers/application_helper.rb b/cable-bench/app/helpers/application_helper.rb new file mode 100644 index 0000000..de6be79 --- /dev/null +++ b/cable-bench/app/helpers/application_helper.rb @@ -0,0 +1,2 @@ +module ApplicationHelper +end diff --git a/cable-bench/app/javascript/application.js b/cable-bench/app/javascript/application.js new file mode 100644 index 0000000..0d7b494 --- /dev/null +++ b/cable-bench/app/javascript/application.js @@ -0,0 +1,3 @@ +// Configure your import map in config/importmap.rb. Read more: https://github.com/rails/importmap-rails +import "@hotwired/turbo-rails" +import "controllers" diff --git a/cable-bench/app/javascript/controllers/application.js b/cable-bench/app/javascript/controllers/application.js new file mode 100644 index 0000000..1213e85 --- /dev/null +++ b/cable-bench/app/javascript/controllers/application.js @@ -0,0 +1,9 @@ +import { Application } from "@hotwired/stimulus" + +const application = Application.start() + +// Configure Stimulus development experience +application.debug = false +window.Stimulus = application + +export { application } diff --git a/cable-bench/app/javascript/controllers/hello_controller.js b/cable-bench/app/javascript/controllers/hello_controller.js new file mode 100644 index 0000000..5975c07 --- /dev/null +++ b/cable-bench/app/javascript/controllers/hello_controller.js @@ -0,0 +1,7 @@ +import { Controller } from "@hotwired/stimulus" + +export default class extends Controller { + connect() { + this.element.textContent = "Hello World!" + } +} diff --git a/cable-bench/app/javascript/controllers/index.js b/cable-bench/app/javascript/controllers/index.js new file mode 100644 index 0000000..1156bf8 --- /dev/null +++ b/cable-bench/app/javascript/controllers/index.js @@ -0,0 +1,4 @@ +// Import and register all your controllers from the importmap via controllers/**/*_controller +import { application } from "controllers/application" +import { eagerLoadControllersFrom } from "@hotwired/stimulus-loading" +eagerLoadControllersFrom("controllers", application) diff --git a/cable-bench/app/jobs/application_job.rb b/cable-bench/app/jobs/application_job.rb new file mode 100644 index 0000000..d394c3d --- /dev/null +++ b/cable-bench/app/jobs/application_job.rb @@ -0,0 +1,7 @@ +class ApplicationJob < ActiveJob::Base + # Automatically retry jobs that encountered a deadlock + # retry_on ActiveRecord::Deadlocked + + # Most jobs are safe to ignore if the underlying records are no longer available + # discard_on ActiveJob::DeserializationError +end diff --git a/cable-bench/app/models/application_record.rb b/cable-bench/app/models/application_record.rb new file mode 100644 index 0000000..b63caeb --- /dev/null +++ b/cable-bench/app/models/application_record.rb @@ -0,0 +1,3 @@ +class ApplicationRecord < ActiveRecord::Base + primary_abstract_class +end diff --git a/cable-bench/app/models/concerns/.keep b/cable-bench/app/models/concerns/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench/app/views/layouts/application.html.erb b/cable-bench/app/views/layouts/application.html.erb new file mode 100644 index 0000000..95012f0 --- /dev/null +++ b/cable-bench/app/views/layouts/application.html.erb @@ -0,0 +1,29 @@ + + + + <%= content_for(:title) || "Cable Bench" %> + + + + + <%= csrf_meta_tags %> + <%= csp_meta_tag %> + + <%= yield :head %> + + <%# Enable PWA manifest for installable apps (make sure to enable in config/routes.rb too!) %> + <%#= tag.link rel: "manifest", href: pwa_manifest_path(format: :json) %> + + + + + + <%# Includes all stylesheet files in app/assets/stylesheets %> + <%= stylesheet_link_tag :app, "data-turbo-track": "reload" %> + <%= javascript_importmap_tags %> + + + + <%= yield %> + + diff --git a/cable-bench/app/views/pwa/manifest.json.erb b/cable-bench/app/views/pwa/manifest.json.erb new file mode 100644 index 0000000..9cbe4ae --- /dev/null +++ b/cable-bench/app/views/pwa/manifest.json.erb @@ -0,0 +1,22 @@ +{ + "name": "CableBench", + "icons": [ + { + "src": "/icon.png", + "type": "image/png", + "sizes": "512x512" + }, + { + "src": "/icon.png", + "type": "image/png", + "sizes": "512x512", + "purpose": "maskable" + } + ], + "start_url": "/", + "display": "standalone", + "scope": "/", + "description": "CableBench.", + "theme_color": "red", + "background_color": "red" +} diff --git a/cable-bench/app/views/pwa/service-worker.js b/cable-bench/app/views/pwa/service-worker.js new file mode 100644 index 0000000..b3a13fb --- /dev/null +++ b/cable-bench/app/views/pwa/service-worker.js @@ -0,0 +1,26 @@ +// Add a service worker for processing Web Push notifications: +// +// self.addEventListener("push", async (event) => { +// const { title, options } = await event.data.json() +// event.waitUntil(self.registration.showNotification(title, options)) +// }) +// +// self.addEventListener("notificationclick", function(event) { +// event.notification.close() +// event.waitUntil( +// clients.matchAll({ type: "window" }).then((clientList) => { +// for (let i = 0; i < clientList.length; i++) { +// let client = clientList[i] +// let clientPath = (new URL(client.url)).pathname +// +// if (clientPath == event.notification.data.path && "focus" in client) { +// return client.focus() +// } +// } +// +// if (clients.openWindow) { +// return clients.openWindow(event.notification.data.path) +// } +// }) +// ) +// }) diff --git a/cable-bench/bin/bench-entrypoint b/cable-bench/bin/bench-entrypoint new file mode 100755 index 0000000..f5cebde --- /dev/null +++ b/cable-bench/bin/bench-entrypoint @@ -0,0 +1,33 @@ +#!/bin/bash +# One image, three benchmark targets. BENCH_MODE selects the cable adapter and +# the process to run: +# solidcable (default) -> Puma serving /cable, Solid Cable (DB polling) +# actioncable -> Puma serving /cable, Action Cable Redis adapter +# anycable -> AnyCable gRPC RPC server (anycable-go is separate) +set -e + +export SECRET_KEY_BASE_DUMMY="${SECRET_KEY_BASE_DUMMY:-1}" + +MODE="${BENCH_MODE:-solidcable}" +case "$MODE" in + actioncable) export CABLE_ADAPTER=redis ;; + solidcable) export CABLE_ADAPTER=solid_cable ;; + anycable) export CABLE_ADAPTER=any_cable ;; + *) echo "[entrypoint] unknown BENCH_MODE='$MODE' (actioncable|solidcable|anycable)" >&2; exit 1 ;; +esac +echo "[entrypoint] BENCH_MODE=$MODE CABLE_ADAPTER=$CABLE_ADAPTER PORT=${PORT:-3000}" + +# Create + load the SQLite databases (primary and, for solid_cable, the cable +# stream table). Idempotent across restarts. +./bin/rails db:prepare + +if [ "$MODE" = "anycable" ]; then + # Bind the RPC server on IPv6 so the anycable-go gateway can dial it over + # Railway's private network. Override ANYCABLE_RPC_HOST in deploy if needed. + export ANYCABLE_RPC_HOST="${ANYCABLE_RPC_HOST:-[::]:50051}" + echo "[entrypoint] starting AnyCable RPC server on $ANYCABLE_RPC_HOST" + exec bundle exec anycable +else + echo "[entrypoint] starting Puma" + exec ./bin/rails server -e production +fi diff --git a/cable-bench/bin/brakeman b/cable-bench/bin/brakeman new file mode 100755 index 0000000..ace1c9b --- /dev/null +++ b/cable-bench/bin/brakeman @@ -0,0 +1,7 @@ +#!/usr/bin/env ruby +require "rubygems" +require "bundler/setup" + +ARGV.unshift("--ensure-latest") + +load Gem.bin_path("brakeman", "brakeman") diff --git a/cable-bench/bin/bundler-audit b/cable-bench/bin/bundler-audit new file mode 100755 index 0000000..e2ef226 --- /dev/null +++ b/cable-bench/bin/bundler-audit @@ -0,0 +1,6 @@ +#!/usr/bin/env ruby +require_relative "../config/boot" +require "bundler/audit/cli" + +ARGV.concat %w[ --config config/bundler-audit.yml ] if ARGV.empty? || ARGV.include?("check") +Bundler::Audit::CLI.start diff --git a/cable-bench/bin/ci b/cable-bench/bin/ci new file mode 100755 index 0000000..4137ad5 --- /dev/null +++ b/cable-bench/bin/ci @@ -0,0 +1,6 @@ +#!/usr/bin/env ruby +require_relative "../config/boot" +require "active_support/continuous_integration" + +CI = ActiveSupport::ContinuousIntegration +require_relative "../config/ci.rb" diff --git a/cable-bench/bin/dev b/cable-bench/bin/dev new file mode 100755 index 0000000..5f91c20 --- /dev/null +++ b/cable-bench/bin/dev @@ -0,0 +1,2 @@ +#!/usr/bin/env ruby +exec "./bin/rails", "server", *ARGV diff --git a/cable-bench/bin/importmap b/cable-bench/bin/importmap new file mode 100755 index 0000000..36502ab --- /dev/null +++ b/cable-bench/bin/importmap @@ -0,0 +1,4 @@ +#!/usr/bin/env ruby + +require_relative "../config/application" +require "importmap/commands" diff --git a/cable-bench/bin/jobs b/cable-bench/bin/jobs new file mode 100755 index 0000000..dcf59f3 --- /dev/null +++ b/cable-bench/bin/jobs @@ -0,0 +1,6 @@ +#!/usr/bin/env ruby + +require_relative "../config/environment" +require "solid_queue/cli" + +SolidQueue::Cli.start(ARGV) diff --git a/cable-bench/bin/rails b/cable-bench/bin/rails new file mode 100755 index 0000000..efc0377 --- /dev/null +++ b/cable-bench/bin/rails @@ -0,0 +1,4 @@ +#!/usr/bin/env ruby +APP_PATH = File.expand_path("../config/application", __dir__) +require_relative "../config/boot" +require "rails/commands" diff --git a/cable-bench/bin/rake b/cable-bench/bin/rake new file mode 100755 index 0000000..4fbf10b --- /dev/null +++ b/cable-bench/bin/rake @@ -0,0 +1,4 @@ +#!/usr/bin/env ruby +require_relative "../config/boot" +require "rake" +Rake.application.run diff --git a/cable-bench/bin/rubocop b/cable-bench/bin/rubocop new file mode 100755 index 0000000..5a20504 --- /dev/null +++ b/cable-bench/bin/rubocop @@ -0,0 +1,8 @@ +#!/usr/bin/env ruby +require "rubygems" +require "bundler/setup" + +# Explicit RuboCop config increases performance slightly while avoiding config confusion. +ARGV.unshift("--config", File.expand_path("../.rubocop.yml", __dir__)) + +load Gem.bin_path("rubocop", "rubocop") diff --git a/cable-bench/bin/setup b/cable-bench/bin/setup new file mode 100755 index 0000000..81be011 --- /dev/null +++ b/cable-bench/bin/setup @@ -0,0 +1,35 @@ +#!/usr/bin/env ruby +require "fileutils" + +APP_ROOT = File.expand_path("..", __dir__) + +def system!(*args) + system(*args, exception: true) +end + +FileUtils.chdir APP_ROOT do + # This script is a way to set up or update your development environment automatically. + # This script is idempotent, so that you can run it at any time and get an expectable outcome. + # Add necessary setup steps to this file. + + puts "== Installing dependencies ==" + system("bundle check") || system!("bundle install") + + # puts "\n== Copying sample files ==" + # unless File.exist?("config/database.yml") + # FileUtils.cp "config/database.yml.sample", "config/database.yml" + # end + + puts "\n== Preparing database ==" + system! "bin/rails db:prepare" + system! "bin/rails db:reset" if ARGV.include?("--reset") + + puts "\n== Removing old logs and tempfiles ==" + system! "bin/rails log:clear tmp:clear" + + unless ARGV.include?("--skip-server") + puts "\n== Starting development server ==" + STDOUT.flush # flush the output before exec(2) so that it displays + exec "bin/dev" + end +end diff --git a/cable-bench/bin/thrust b/cable-bench/bin/thrust new file mode 100755 index 0000000..36bde2d --- /dev/null +++ b/cable-bench/bin/thrust @@ -0,0 +1,5 @@ +#!/usr/bin/env ruby +require "rubygems" +require "bundler/setup" + +load Gem.bin_path("thruster", "thrust") diff --git a/cable-bench/config.ru b/cable-bench/config.ru new file mode 100644 index 0000000..4a3c09a --- /dev/null +++ b/cable-bench/config.ru @@ -0,0 +1,6 @@ +# This file is used by Rack-based servers to start the application. + +require_relative "config/environment" + +run Rails.application +Rails.application.load_server diff --git a/cable-bench/config/anycable.yml b/cable-bench/config/anycable.yml new file mode 100644 index 0000000..b2ebab2 --- /dev/null +++ b/cable-bench/config/anycable.yml @@ -0,0 +1,17 @@ +# AnyCable (RPC) configuration for the AnyCable benchmark mode only. The other +# two modes (solid_cable, redis) never load this — Rails serves /cable itself. +# +# Here Rails runs the gRPC RPC server (`bundle exec anycable`) and anycable-go +# is the separate WebSocket gateway. Broadcasts go over Redis Streams (redisx) +# so the gateway's broker can keep stream history and resume the extended +# Action Cable protocol on reconnect (the delivery-guarantee path). Wire the +# Redis URL and RPC bind via env in deployment (ANYCABLE_REDIS_URL, +# ANYCABLE_RPC_HOST). +default: &default + broadcast_adapter: redisx + +development: + <<: *default + +production: + <<: *default diff --git a/cable-bench/config/application.rb b/cable-bench/config/application.rb new file mode 100644 index 0000000..0aeaddf --- /dev/null +++ b/cable-bench/config/application.rb @@ -0,0 +1,42 @@ +require_relative "boot" + +require "rails" +# Pick the frameworks you want: +require "active_model/railtie" +require "active_job/railtie" +require "active_record/railtie" +# require "active_storage/engine" +require "action_controller/railtie" +# require "action_mailer/railtie" +# require "action_mailbox/engine" +# require "action_text/engine" +require "action_view/railtie" +require "action_cable/engine" +# require "rails/test_unit/railtie" + +# Require the gems listed in Gemfile, including any gems +# you've limited to :test, :development, or :production. +Bundler.require(*Rails.groups) + +module CableBench + class Application < Rails::Application + # Initialize configuration defaults for originally generated Rails version. + config.load_defaults 8.1 + + # Please, add to the `ignore` list any other `lib` subdirectories that do + # not contain `.rb` files, or that should not be reloaded or eager loaded. + # Common ones are `templates`, `generators`, or `middleware`, for example. + config.autoload_lib(ignore: %w[assets tasks]) + + # Configuration for the application, engines, and railties goes here. + # + # These settings can be overridden in specific environments using the files + # in config/environments, which are processed later. + # + # config.time_zone = "Central Time (US & Canada)" + # config.eager_load_paths << Rails.root.join("extras") + + # Don't generate system test files. + config.generators.system_tests = nil + end +end diff --git a/cable-bench/config/boot.rb b/cable-bench/config/boot.rb new file mode 100644 index 0000000..988a5dd --- /dev/null +++ b/cable-bench/config/boot.rb @@ -0,0 +1,4 @@ +ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__) + +require "bundler/setup" # Set up gems listed in the Gemfile. +require "bootsnap/setup" # Speed up boot time by caching expensive operations. diff --git a/cable-bench/config/bundler-audit.yml b/cable-bench/config/bundler-audit.yml new file mode 100644 index 0000000..e74b3af --- /dev/null +++ b/cable-bench/config/bundler-audit.yml @@ -0,0 +1,5 @@ +# Audit all gems listed in the Gemfile for known security problems by running bin/bundler-audit. +# CVEs that are not relevant to the application can be enumerated on the ignore list below. + +ignore: + - CVE-THAT-DOES-NOT-APPLY diff --git a/cable-bench/config/cable.yml b/cable-bench/config/cable.yml new file mode 100644 index 0000000..b7b385a --- /dev/null +++ b/cable-bench/config/cable.yml @@ -0,0 +1,29 @@ +# Three benchmark adapters selected at boot by CABLE_ADAPTER: +# solid_cable (default) — Rails 8 out-of-the-box, DB-polling, in-process Puma +# redis — classic Action Cable pub/sub, in-process Puma +# any_cable — out-of-process anycable-go gateway, Rails over gRPC +# +# polling_interval / message_retention can be tuned via env for the Solid Cable +# runs without touching code. +<% adapter = ENV.fetch("CABLE_ADAPTER", "solid_cable") %> +development: + adapter: async + +test: + adapter: test + +production: +<% if adapter == "redis" %> + adapter: redis + url: <%= ENV.fetch("REDIS_URL", "redis://localhost:6379/1") %> + channel_prefix: cable_bench +<% elsif adapter == "any_cable" %> + adapter: any_cable +<% else %> + adapter: solid_cable + connects_to: + database: + writing: cable + polling_interval: <%= ENV.fetch("SOLID_CABLE_POLLING_INTERVAL", "0.1") %>.seconds + message_retention: <%= ENV.fetch("SOLID_CABLE_RETENTION", "1.day") %> +<% end %> diff --git a/cable-bench/config/cache.yml b/cable-bench/config/cache.yml new file mode 100644 index 0000000..19d4908 --- /dev/null +++ b/cable-bench/config/cache.yml @@ -0,0 +1,16 @@ +default: &default + store_options: + # Cap age of oldest cache entry to fulfill retention policies + # max_age: <%= 60.days.to_i %> + max_size: <%= 256.megabytes %> + namespace: <%= Rails.env %> + +development: + <<: *default + +test: + <<: *default + +production: + database: cache + <<: *default diff --git a/cable-bench/config/ci.rb b/cable-bench/config/ci.rb new file mode 100644 index 0000000..239b343 --- /dev/null +++ b/cable-bench/config/ci.rb @@ -0,0 +1,20 @@ +# Run using bin/ci + +CI.run do + step "Setup", "bin/setup --skip-server" + + step "Style: Ruby", "bin/rubocop" + + step "Security: Gem audit", "bin/bundler-audit" + step "Security: Importmap vulnerability audit", "bin/importmap audit" + step "Security: Brakeman code analysis", "bin/brakeman --quiet --no-pager --exit-on-warn --exit-on-error" + + + # Optional: set a green GitHub commit status to unblock PR merge. + # Requires the `gh` CLI and `gh extension install basecamp/gh-signoff`. + # if success? + # step "Signoff: All systems go. Ready for merge and deploy.", "gh signoff" + # else + # failure "Signoff: CI failed. Do not merge or deploy.", "Fix the issues and try again." + # end +end diff --git a/cable-bench/config/credentials.yml.enc b/cable-bench/config/credentials.yml.enc new file mode 100644 index 0000000..1d889f4 --- /dev/null +++ b/cable-bench/config/credentials.yml.enc @@ -0,0 +1 @@ +eagm44xPbLb4jZgrmBiNBwIRQM+rbzjDEq8VpI1j0dSmvihIBuvdsO5hGvn88a9kr8ijIThnuR3el+OX7M8RLvfImeDVz6MgKj95zBAkHGwbi5CgMzjEd4jR2XyDdCxYpqi9WD1M3PAMymdHJpheREPMhOvF0CQ/Ac/57VKqiqOKybCSZGV7G30XOubAruk+kGKTdGEWgsMlTqhH+qbpDHP+ktn9Gu/v7LEoKkxCL1ETA6E53/6mwofWwcXuTClxn0oHkrOkm6NU4q6TfDBvTSPxuPVG600t29hPeBPt9RH/fa8ytsVpdbe7RUNobSr87bi7F5QudH0MOJBRezCou2LxeZgneokS7QJTh1/QhY572+UU3YePJhxyCukDIp07gewWOGPzz24/h/XNcyKLpUf9KIEGFQ7OQ++WcIIp+JSU/2yUCc+TiT0amnubSYOYjfzfPhKVydEYCkMOuH/2+ptJC8n7ETrp2L0v5pnXv732Cxnsgp3o2K96--5zE13fNJAxth31ZH--93XIYtVm7Aub41gBda9/Xw== \ No newline at end of file diff --git a/cable-bench/config/database.yml b/cable-bench/config/database.yml new file mode 100644 index 0000000..4e71461 --- /dev/null +++ b/cable-bench/config/database.yml @@ -0,0 +1,48 @@ +# SQLite. Versions 3.8.0 and up are supported. +# gem install sqlite3 +# +# Ensure the SQLite 3 gem is defined in your Gemfile +# gem "sqlite3" +# +default: &default + adapter: sqlite3 + max_connections: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %> + timeout: 5000 + +development: + <<: *default + database: storage/development.sqlite3 + +# Warning: The database defined as "test" will be erased and +# re-generated from your development database when you run "rake". +# Do not set this db to the same as development or production. +test: + <<: *default + database: storage/test.sqlite3 + +# SQLite3 write its data on the local filesystem, as such it requires +# persistent disks. If you are deploying to a managed service, you should +# make sure it provides disk persistence, as many don't. +# +# Similarly, if you deploy your application as a Docker container, you must +# ensure the database is located in a persisted volume. +# Benchmark data is disposable: the SQLite files live on the container's +# ephemeral disk and are recreated by `db:prepare` on every boot. The cable DB +# is the one that matters (Solid Cable's message stream); cache/queue are +# unused but kept so db:prepare succeeds. +production: + primary: + <<: *default + database: storage/production.sqlite3 + cache: + <<: *default + database: storage/production_cache.sqlite3 + migrations_paths: db/cache_migrate + queue: + <<: *default + database: storage/production_queue.sqlite3 + migrations_paths: db/queue_migrate + cable: + <<: *default + database: storage/production_cable.sqlite3 + migrations_paths: db/cable_migrate diff --git a/cable-bench/config/environment.rb b/cable-bench/config/environment.rb new file mode 100644 index 0000000..cac5315 --- /dev/null +++ b/cable-bench/config/environment.rb @@ -0,0 +1,5 @@ +# Load the Rails application. +require_relative "application" + +# Initialize the Rails application. +Rails.application.initialize! diff --git a/cable-bench/config/environments/development.rb b/cable-bench/config/environments/development.rb new file mode 100644 index 0000000..64dd9a0 --- /dev/null +++ b/cable-bench/config/environments/development.rb @@ -0,0 +1,71 @@ +require "active_support/core_ext/integer/time" + +Rails.application.configure do + # Settings specified here will take precedence over those in config/application.rb. + + # Make code changes take effect immediately without server restart. + config.enable_reloading = true + + # The load generator opens raw WebSockets with no Origin header (same as in + # production). Skip the Action Cable origin check so local smoke tests and + # the bench-runner can connect. + config.action_cable.disable_request_forgery_protection = true + + # Do not eager load code on boot. + config.eager_load = false + + # Show full error reports. + config.consider_all_requests_local = true + + # Enable server timing. + config.server_timing = true + + # Enable/disable Action Controller caching. By default Action Controller caching is disabled. + # Run rails dev:cache to toggle Action Controller caching. + if Rails.root.join("tmp/caching-dev.txt").exist? + config.action_controller.perform_caching = true + config.action_controller.enable_fragment_cache_logging = true + config.public_file_server.headers = { "cache-control" => "public, max-age=#{2.days.to_i}" } + else + config.action_controller.perform_caching = false + end + + # Change to :null_store to avoid any caching. + config.cache_store = :memory_store + + # Print deprecation notices to the Rails logger. + config.active_support.deprecation = :log + + # Raise an error on page load if there are pending migrations. + config.active_record.migration_error = :page_load + + # Highlight code that triggered database queries in logs. + config.active_record.verbose_query_logs = true + + # Append comments with runtime information tags to SQL queries in logs. + config.active_record.query_log_tags_enabled = true + + # Highlight code that enqueued background job in logs. + config.active_job.verbose_enqueue_logs = true + + # Highlight code that triggered redirect in logs. + config.action_dispatch.verbose_redirect_logs = true + + # Suppress logger output for asset requests. + config.assets.quiet = true + + # Raises error for missing translations. + # config.i18n.raise_on_missing_translations = true + + # Annotate rendered view with file names. + config.action_view.annotate_rendered_view_with_filenames = true + + # Uncomment if you wish to allow Action Cable access from any origin. + # config.action_cable.disable_request_forgery_protection = true + + # Raise error when a before_action's only/except options reference missing actions. + config.action_controller.raise_on_missing_callback_actions = true + + # Apply autocorrection by RuboCop to files generated by `bin/rails generate`. + # config.generators.apply_rubocop_autocorrect_after_generate! +end diff --git a/cable-bench/config/environments/production.rb b/cable-bench/config/environments/production.rb new file mode 100644 index 0000000..df867ef --- /dev/null +++ b/cable-bench/config/environments/production.rb @@ -0,0 +1,75 @@ +require "active_support/core_ext/integer/time" + +Rails.application.configure do + # Settings specified here will take precedence over those in config/application.rb. + + # Code is not reloaded between requests. + config.enable_reloading = false + + # Eager load code on boot for better performance and memory savings (ignored by Rake tasks). + config.eager_load = true + + # Full error reports are disabled. + config.consider_all_requests_local = false + + # Turn on fragment caching in view templates. + config.action_controller.perform_caching = true + + # Cache assets for far-future expiry since they are all digest stamped. + config.public_file_server.headers = { "cache-control" => "public, max-age=#{1.year.to_i}" } + + # Enable serving of images, stylesheets, and JavaScripts from an asset server. + # config.asset_host = "http://assets.example.com" + + # This app only ever receives traffic from the bench-runner fleet over + # Railway's private network (plain HTTP/WS, no TLS-terminating proxy in + # front), so SSL enforcement would just break internal connections. + config.assume_ssl = false + config.force_ssl = false + + # The load generator opens raw WebSockets without a matching Origin header; + # there is no browser session to protect, so skip the Action Cable origin check. + config.action_cable.disable_request_forgery_protection = true + + # Skip http-to-https redirect for the default health check endpoint. + # config.ssl_options = { redirect: { exclude: ->(request) { request.path == "/up" } } } + + # Log to STDOUT with the current request id as a default log tag. + config.log_tags = [ :request_id ] + config.logger = ActiveSupport::TaggedLogging.logger(STDOUT) + + # Change to "debug" to log everything (including potentially personally-identifiable information!). + config.log_level = ENV.fetch("RAILS_LOG_LEVEL", "info") + + # Prevent health checks from clogging up the logs. + config.silence_healthcheck_path = "/up" + + # Don't log any deprecations. + config.active_support.report_deprecations = false + + # Replace the default in-process memory cache store with a durable alternative. + config.cache_store = :solid_cache_store + + # Replace the default in-process and non-durable queuing backend for Active Job. + config.active_job.queue_adapter = :solid_queue + config.solid_queue.connects_to = { database: { writing: :queue } } + + # Enable locale fallbacks for I18n (makes lookups for any locale fall back to + # the I18n.default_locale when a translation cannot be found). + config.i18n.fallbacks = true + + # Do not dump schema after migrations. + config.active_record.dump_schema_after_migration = false + + # Only use :id for inspections in production. + config.active_record.attributes_for_inspect = [ :id ] + + # Enable DNS rebinding protection and other `Host` header attacks. + # config.hosts = [ + # "example.com", # Allow requests from example.com + # /.*\.example\.com/ # Allow requests from subdomains like `www.example.com` + # ] + # + # Skip DNS rebinding protection for the default health check endpoint. + # config.host_authorization = { exclude: ->(request) { request.path == "/up" } } +end diff --git a/cable-bench/config/environments/test.rb b/cable-bench/config/environments/test.rb new file mode 100644 index 0000000..14bc29e --- /dev/null +++ b/cable-bench/config/environments/test.rb @@ -0,0 +1,42 @@ +# The test environment is used exclusively to run your application's +# test suite. You never need to work with it otherwise. Remember that +# your test database is "scratch space" for the test suite and is wiped +# and recreated between test runs. Don't rely on the data there! + +Rails.application.configure do + # Settings specified here will take precedence over those in config/application.rb. + + # While tests run files are not watched, reloading is not necessary. + config.enable_reloading = false + + # Eager loading loads your entire application. When running a single test locally, + # this is usually not necessary, and can slow down your test suite. However, it's + # recommended that you enable it in continuous integration systems to ensure eager + # loading is working properly before deploying your code. + config.eager_load = ENV["CI"].present? + + # Configure public file server for tests with cache-control for performance. + config.public_file_server.headers = { "cache-control" => "public, max-age=3600" } + + # Show full error reports. + config.consider_all_requests_local = true + config.cache_store = :null_store + + # Render exception templates for rescuable exceptions and raise for other exceptions. + config.action_dispatch.show_exceptions = :rescuable + + # Disable request forgery protection in test environment. + config.action_controller.allow_forgery_protection = false + + # Print deprecation notices to the stderr. + config.active_support.deprecation = :stderr + + # Raises error for missing translations. + # config.i18n.raise_on_missing_translations = true + + # Annotate rendered view with file names. + # config.action_view.annotate_rendered_view_with_filenames = true + + # Raise error when a before_action's only/except options reference missing actions. + config.action_controller.raise_on_missing_callback_actions = true +end diff --git a/cable-bench/config/importmap.rb b/cable-bench/config/importmap.rb new file mode 100644 index 0000000..909dfc5 --- /dev/null +++ b/cable-bench/config/importmap.rb @@ -0,0 +1,7 @@ +# Pin npm packages by running ./bin/importmap + +pin "application" +pin "@hotwired/turbo-rails", to: "turbo.min.js" +pin "@hotwired/stimulus", to: "stimulus.min.js" +pin "@hotwired/stimulus-loading", to: "stimulus-loading.js" +pin_all_from "app/javascript/controllers", under: "controllers" diff --git a/cable-bench/config/initializers/assets.rb b/cable-bench/config/initializers/assets.rb new file mode 100644 index 0000000..4873244 --- /dev/null +++ b/cable-bench/config/initializers/assets.rb @@ -0,0 +1,7 @@ +# Be sure to restart your server when you modify this file. + +# Version of your assets, change this if you want to expire all your assets. +Rails.application.config.assets.version = "1.0" + +# Add additional assets to the asset load path. +# Rails.application.config.assets.paths << Emoji.images_path diff --git a/cable-bench/config/initializers/content_security_policy.rb b/cable-bench/config/initializers/content_security_policy.rb new file mode 100644 index 0000000..d51d713 --- /dev/null +++ b/cable-bench/config/initializers/content_security_policy.rb @@ -0,0 +1,29 @@ +# Be sure to restart your server when you modify this file. + +# Define an application-wide content security policy. +# See the Securing Rails Applications Guide for more information: +# https://guides.rubyonrails.org/security.html#content-security-policy-header + +# Rails.application.configure do +# config.content_security_policy do |policy| +# policy.default_src :self, :https +# policy.font_src :self, :https, :data +# policy.img_src :self, :https, :data +# policy.object_src :none +# policy.script_src :self, :https +# policy.style_src :self, :https +# # Specify URI for violation reports +# # policy.report_uri "/csp-violation-report-endpoint" +# end +# +# # Generate session nonces for permitted importmap, inline scripts, and inline styles. +# config.content_security_policy_nonce_generator = ->(request) { request.session.id.to_s } +# config.content_security_policy_nonce_directives = %w(script-src style-src) +# +# # Automatically add `nonce` to `javascript_tag`, `javascript_include_tag`, and `stylesheet_link_tag` +# # if the corresponding directives are specified in `content_security_policy_nonce_directives`. +# # config.content_security_policy_nonce_auto = true +# +# # Report violations without enforcing the policy. +# # config.content_security_policy_report_only = true +# end diff --git a/cable-bench/config/initializers/filter_parameter_logging.rb b/cable-bench/config/initializers/filter_parameter_logging.rb new file mode 100644 index 0000000..c0b717f --- /dev/null +++ b/cable-bench/config/initializers/filter_parameter_logging.rb @@ -0,0 +1,8 @@ +# Be sure to restart your server when you modify this file. + +# Configure parameters to be partially matched (e.g. passw matches password) and filtered from the log file. +# Use this to limit dissemination of sensitive information. +# See the ActiveSupport::ParameterFilter documentation for supported notations and behaviors. +Rails.application.config.filter_parameters += [ + :passw, :email, :secret, :token, :_key, :crypt, :salt, :certificate, :otp, :ssn, :cvv, :cvc +] diff --git a/cable-bench/config/initializers/inflections.rb b/cable-bench/config/initializers/inflections.rb new file mode 100644 index 0000000..3860f65 --- /dev/null +++ b/cable-bench/config/initializers/inflections.rb @@ -0,0 +1,16 @@ +# Be sure to restart your server when you modify this file. + +# Add new inflection rules using the following format. Inflections +# are locale specific, and you may define rules for as many different +# locales as you wish. All of these examples are active by default: +# ActiveSupport::Inflector.inflections(:en) do |inflect| +# inflect.plural /^(ox)$/i, "\\1en" +# inflect.singular /^(ox)en/i, "\\1" +# inflect.irregular "person", "people" +# inflect.uncountable %w( fish sheep ) +# end + +# These inflection rules are supported but not enabled by default: +# ActiveSupport::Inflector.inflections(:en) do |inflect| +# inflect.acronym "RESTful" +# end diff --git a/cable-bench/config/locales/en.yml b/cable-bench/config/locales/en.yml new file mode 100644 index 0000000..6c349ae --- /dev/null +++ b/cable-bench/config/locales/en.yml @@ -0,0 +1,31 @@ +# Files in the config/locales directory are used for internationalization and +# are automatically loaded by Rails. If you want to use locales other than +# English, add the necessary files in this directory. +# +# To use the locales, use `I18n.t`: +# +# I18n.t "hello" +# +# In views, this is aliased to just `t`: +# +# <%= t("hello") %> +# +# To use a different locale, set it with `I18n.locale`: +# +# I18n.locale = :es +# +# This would use the information in config/locales/es.yml. +# +# To learn more about the API, please read the Rails Internationalization guide +# at https://guides.rubyonrails.org/i18n.html. +# +# Be aware that YAML interprets the following case-insensitive strings as +# booleans: `true`, `false`, `on`, `off`, `yes`, `no`. Therefore, these strings +# must be quoted to be interpreted as strings. For example: +# +# en: +# "yes": yup +# enabled: "ON" + +en: + hello: "Hello world" diff --git a/cable-bench/config/puma.rb b/cable-bench/config/puma.rb new file mode 100644 index 0000000..f78c735 --- /dev/null +++ b/cable-bench/config/puma.rb @@ -0,0 +1,52 @@ +# This configuration file will be evaluated by Puma. The top-level methods that +# are invoked here are part of Puma's configuration DSL. For more information +# about methods provided by the DSL, see https://puma.io/puma/Puma/DSL.html. +# +# Puma starts a configurable number of processes (workers) and each process +# serves each request in a thread from an internal thread pool. +# +# You can control the number of workers using ENV["WEB_CONCURRENCY"]. You +# should only set this value when you want to run 2 or more workers. The +# default is already 1. You can set it to `auto` to automatically start a worker +# for each available processor. +# +# The ideal number of threads per worker depends both on how much time the +# application spends waiting for IO operations and on how much you wish to +# prioritize throughput over latency. +# +# As a rule of thumb, increasing the number of threads will increase how much +# traffic a given process can handle (throughput), but due to CRuby's +# Global VM Lock (GVL) it has diminishing returns and will degrade the +# response time (latency) of the application. +# +# The default is set to 3 threads as it's deemed a decent compromise between +# throughput and latency for the average Rails application. +# +# Any libraries that use a connection pool or another resource pool should +# be configured to provide at least as many connections as the number of +# threads. This includes Active Record's `pool` parameter in `database.yml`. +threads_count = ENV.fetch("RAILS_MAX_THREADS", 3) +threads threads_count, threads_count + +# Run WEB_CONCURRENCY worker processes (default 1). The stock Rails puma.rb +# omits this directive, so WEB_CONCURRENCY was silently ignored and Puma ran a +# single process regardless. We set it explicitly so the Action Cable target's +# process count matches the Falcon target's `falcon serve --count` for an +# apples-to-apples WS-engine comparison. +workers ENV.fetch("WEB_CONCURRENCY", 1).to_i + +# Bind IPv6 dual-stack (`[::]`) rather than the default IPv4 `0.0.0.0`: Railway's +# private network routes over IPv6, so internal clients (the bench-runner fleet) +# can only reach Puma on `[::]`. On Linux this also accepts IPv4, and it works +# locally on macOS too. A single bind avoids an address-in-use clash with `port`. +bind "tcp://[::]:#{ENV.fetch('PORT', 3000)}" + +# Allow puma to be restarted by `bin/rails restart` command. +plugin :tmp_restart + +# Run the Solid Queue supervisor inside of Puma for single-server deployments. +plugin :solid_queue if ENV["SOLID_QUEUE_IN_PUMA"] + +# Specify the PID file. Defaults to tmp/pids/server.pid in development. +# In other environments, only set the PID file if requested. +pidfile ENV["PIDFILE"] if ENV["PIDFILE"] diff --git a/cable-bench/config/queue.yml b/cable-bench/config/queue.yml new file mode 100644 index 0000000..9eace59 --- /dev/null +++ b/cable-bench/config/queue.yml @@ -0,0 +1,18 @@ +default: &default + dispatchers: + - polling_interval: 1 + batch_size: 500 + workers: + - queues: "*" + threads: 3 + processes: <%= ENV.fetch("JOB_CONCURRENCY", 1) %> + polling_interval: 0.1 + +development: + <<: *default + +test: + <<: *default + +production: + <<: *default diff --git a/cable-bench/config/recurring.yml b/cable-bench/config/recurring.yml new file mode 100644 index 0000000..b4207f9 --- /dev/null +++ b/cable-bench/config/recurring.yml @@ -0,0 +1,15 @@ +# examples: +# periodic_cleanup: +# class: CleanSoftDeletedRecordsJob +# queue: background +# args: [ 1000, { batch_size: 500 } ] +# schedule: every hour +# periodic_cleanup_with_command: +# command: "SoftDeletedRecord.due.delete_all" +# priority: 2 +# schedule: at 5am every day + +production: + clear_solid_queue_finished_jobs: + command: "SolidQueue::Job.clear_finished_in_batches(sleep_between_batches: 0.3)" + schedule: every hour at minute 12 diff --git a/cable-bench/config/routes.rb b/cable-bench/config/routes.rb new file mode 100644 index 0000000..b384a11 --- /dev/null +++ b/cable-bench/config/routes.rb @@ -0,0 +1,23 @@ +Rails.application.routes.draw do + # Define your application routes per the DSL in https://guides.rubyonrails.org/routing.html + + # Reveal health status on /up that returns 200 if the app boots with no exceptions, otherwise 500. + # Can be used by load balancers and uptime monitors to verify that the app is live. + get "up" => "rails/health#show", as: :rails_health_check + + # Benchmark harness surface. + get "health" => "bench#health" + post "_bench/broadcast" => "bench#broadcast" + + # Serve Action Cable in-process for the solid_cable / redis modes. In + # anycable mode the Rails process runs the gRPC server (not Puma), so this + # mount is never hit and stays harmless. + mount ActionCable.server => "/cable" + + # Render dynamic PWA files from app/views/pwa/* (remember to link manifest in application.html.erb) + # get "manifest" => "rails/pwa#manifest", as: :pwa_manifest + # get "service-worker" => "rails/pwa#service_worker", as: :pwa_service_worker + + # Defines the root path route ("/") + # root "posts#index" +end diff --git a/cable-bench/db/cable_schema.rb b/cable-bench/db/cable_schema.rb new file mode 100644 index 0000000..2366660 --- /dev/null +++ b/cable-bench/db/cable_schema.rb @@ -0,0 +1,11 @@ +ActiveRecord::Schema[7.1].define(version: 1) do + create_table "solid_cable_messages", force: :cascade do |t| + t.binary "channel", limit: 1024, null: false + t.binary "payload", limit: 536870912, null: false + t.datetime "created_at", null: false + t.integer "channel_hash", limit: 8, null: false + t.index ["channel"], name: "index_solid_cable_messages_on_channel" + t.index ["channel_hash"], name: "index_solid_cable_messages_on_channel_hash" + t.index ["created_at"], name: "index_solid_cable_messages_on_created_at" + end +end diff --git a/cable-bench/db/cache_schema.rb b/cable-bench/db/cache_schema.rb new file mode 100644 index 0000000..81a410d --- /dev/null +++ b/cable-bench/db/cache_schema.rb @@ -0,0 +1,12 @@ +ActiveRecord::Schema[7.2].define(version: 1) do + create_table "solid_cache_entries", force: :cascade do |t| + t.binary "key", limit: 1024, null: false + t.binary "value", limit: 536870912, null: false + t.datetime "created_at", null: false + t.integer "key_hash", limit: 8, null: false + t.integer "byte_size", limit: 4, null: false + t.index ["byte_size"], name: "index_solid_cache_entries_on_byte_size" + t.index ["key_hash", "byte_size"], name: "index_solid_cache_entries_on_key_hash_and_byte_size" + t.index ["key_hash"], name: "index_solid_cache_entries_on_key_hash", unique: true + end +end diff --git a/cable-bench/db/queue_schema.rb b/cable-bench/db/queue_schema.rb new file mode 100644 index 0000000..85194b6 --- /dev/null +++ b/cable-bench/db/queue_schema.rb @@ -0,0 +1,129 @@ +ActiveRecord::Schema[7.1].define(version: 1) do + create_table "solid_queue_blocked_executions", force: :cascade do |t| + t.bigint "job_id", null: false + t.string "queue_name", null: false + t.integer "priority", default: 0, null: false + t.string "concurrency_key", null: false + t.datetime "expires_at", null: false + t.datetime "created_at", null: false + t.index [ "concurrency_key", "priority", "job_id" ], name: "index_solid_queue_blocked_executions_for_release" + t.index [ "expires_at", "concurrency_key" ], name: "index_solid_queue_blocked_executions_for_maintenance" + t.index [ "job_id" ], name: "index_solid_queue_blocked_executions_on_job_id", unique: true + end + + create_table "solid_queue_claimed_executions", force: :cascade do |t| + t.bigint "job_id", null: false + t.bigint "process_id" + t.datetime "created_at", null: false + t.index [ "job_id" ], name: "index_solid_queue_claimed_executions_on_job_id", unique: true + t.index [ "process_id", "job_id" ], name: "index_solid_queue_claimed_executions_on_process_id_and_job_id" + end + + create_table "solid_queue_failed_executions", force: :cascade do |t| + t.bigint "job_id", null: false + t.text "error" + t.datetime "created_at", null: false + t.index [ "job_id" ], name: "index_solid_queue_failed_executions_on_job_id", unique: true + end + + create_table "solid_queue_jobs", force: :cascade do |t| + t.string "queue_name", null: false + t.string "class_name", null: false + t.text "arguments" + t.integer "priority", default: 0, null: false + t.string "active_job_id" + t.datetime "scheduled_at" + t.datetime "finished_at" + t.string "concurrency_key" + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + t.index [ "active_job_id" ], name: "index_solid_queue_jobs_on_active_job_id" + t.index [ "class_name" ], name: "index_solid_queue_jobs_on_class_name" + t.index [ "finished_at" ], name: "index_solid_queue_jobs_on_finished_at" + t.index [ "queue_name", "finished_at" ], name: "index_solid_queue_jobs_for_filtering" + t.index [ "scheduled_at", "finished_at" ], name: "index_solid_queue_jobs_for_alerting" + end + + create_table "solid_queue_pauses", force: :cascade do |t| + t.string "queue_name", null: false + t.datetime "created_at", null: false + t.index [ "queue_name" ], name: "index_solid_queue_pauses_on_queue_name", unique: true + end + + create_table "solid_queue_processes", force: :cascade do |t| + t.string "kind", null: false + t.datetime "last_heartbeat_at", null: false + t.bigint "supervisor_id" + t.integer "pid", null: false + t.string "hostname" + t.text "metadata" + t.datetime "created_at", null: false + t.string "name", null: false + t.index [ "last_heartbeat_at" ], name: "index_solid_queue_processes_on_last_heartbeat_at" + t.index [ "name", "supervisor_id" ], name: "index_solid_queue_processes_on_name_and_supervisor_id", unique: true + t.index [ "supervisor_id" ], name: "index_solid_queue_processes_on_supervisor_id" + end + + create_table "solid_queue_ready_executions", force: :cascade do |t| + t.bigint "job_id", null: false + t.string "queue_name", null: false + t.integer "priority", default: 0, null: false + t.datetime "created_at", null: false + t.index [ "job_id" ], name: "index_solid_queue_ready_executions_on_job_id", unique: true + t.index [ "priority", "job_id" ], name: "index_solid_queue_poll_all" + t.index [ "queue_name", "priority", "job_id" ], name: "index_solid_queue_poll_by_queue" + end + + create_table "solid_queue_recurring_executions", force: :cascade do |t| + t.bigint "job_id", null: false + t.string "task_key", null: false + t.datetime "run_at", null: false + t.datetime "created_at", null: false + t.index [ "job_id" ], name: "index_solid_queue_recurring_executions_on_job_id", unique: true + t.index [ "task_key", "run_at" ], name: "index_solid_queue_recurring_executions_on_task_key_and_run_at", unique: true + end + + create_table "solid_queue_recurring_tasks", force: :cascade do |t| + t.string "key", null: false + t.string "schedule", null: false + t.string "command", limit: 2048 + t.string "class_name" + t.text "arguments" + t.string "queue_name" + t.integer "priority", default: 0 + t.boolean "static", default: true, null: false + t.text "description" + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + t.index [ "key" ], name: "index_solid_queue_recurring_tasks_on_key", unique: true + t.index [ "static" ], name: "index_solid_queue_recurring_tasks_on_static" + end + + create_table "solid_queue_scheduled_executions", force: :cascade do |t| + t.bigint "job_id", null: false + t.string "queue_name", null: false + t.integer "priority", default: 0, null: false + t.datetime "scheduled_at", null: false + t.datetime "created_at", null: false + t.index [ "job_id" ], name: "index_solid_queue_scheduled_executions_on_job_id", unique: true + t.index [ "scheduled_at", "priority", "job_id" ], name: "index_solid_queue_dispatch_all" + end + + create_table "solid_queue_semaphores", force: :cascade do |t| + t.string "key", null: false + t.integer "value", default: 1, null: false + t.datetime "expires_at", null: false + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + t.index [ "expires_at" ], name: "index_solid_queue_semaphores_on_expires_at" + t.index [ "key", "value" ], name: "index_solid_queue_semaphores_on_key_and_value" + t.index [ "key" ], name: "index_solid_queue_semaphores_on_key", unique: true + end + + add_foreign_key "solid_queue_blocked_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade + add_foreign_key "solid_queue_claimed_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade + add_foreign_key "solid_queue_failed_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade + add_foreign_key "solid_queue_ready_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade + add_foreign_key "solid_queue_recurring_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade + add_foreign_key "solid_queue_scheduled_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade +end diff --git a/cable-bench/db/seeds.rb b/cable-bench/db/seeds.rb new file mode 100644 index 0000000..4fbd6ed --- /dev/null +++ b/cable-bench/db/seeds.rb @@ -0,0 +1,9 @@ +# This file should ensure the existence of records required to run the application in every environment (production, +# development, test). The code here should be idempotent so that it can be executed at any point in every environment. +# The data can then be loaded with the bin/rails db:seed command (or created alongside the database with db:setup). +# +# Example: +# +# ["Action", "Comedy", "Drama", "Horror"].each do |genre_name| +# MovieGenre.find_or_create_by!(name: genre_name) +# end diff --git a/cable-bench/docker-compose.yml b/cable-bench/docker-compose.yml new file mode 100644 index 0000000..4e47859 --- /dev/null +++ b/cable-bench/docker-compose.yml @@ -0,0 +1,56 @@ +# Local validation of the three cable-bench modes against the production image. +# Mirrors the Railway topology so we can shake out config before deploying. +# +# docker compose up --build +# +# Solid Cable: ws://localhost:3001/cable broadcast http://localhost:3001/_bench/broadcast +# Action Cable: ws://localhost:3002/cable broadcast http://localhost:3002/_bench/broadcast +# AnyCable: ws://localhost:8080/cable broadcast http://localhost:8080/_broadcast (via gateway) +# (rails-anycable runs the gRPC backend; anycable-go is the WS gateway) +services: + redis: + image: redis:7-alpine + ports: ["6379:6379"] + + rails-solidcable: + build: . + image: cable-bench:local + environment: + BENCH_MODE: solidcable + PORT: "3000" + ports: ["3001:3000"] + + rails-actioncable: + image: cable-bench:local + environment: + BENCH_MODE: actioncable + PORT: "3000" + REDIS_URL: redis://redis:6379/1 + ports: ["3002:3000"] + depends_on: [redis] + + rails-anycable: + image: cable-bench:local + environment: + BENCH_MODE: anycable + # IPv4 bind for the local docker bridge (Railway uses [::] over its + # IPv6 private network; the entrypoint defaults to that). + ANYCABLE_RPC_HOST: 0.0.0.0:50051 + expose: ["50051"] + + anycable-go: + image: anycable/anycable-go:latest + environment: + ANYCABLE_HOST: "0.0.0.0" + ANYCABLE_PORT: "8080" + ANYCABLE_RPC_HOST: rails-anycable:50051 + # Broker preset + memory broker enables reliable streams (the extended + # protocol resume that backfills messages missed during a disconnect). + ANYCABLE_PRESETS: broker + ANYCABLE_BROKER: memory + ANYCABLE_BROADCAST_ADAPTER: http + # Serve the HTTP broadcast handler on the main port so it's reachable at + # :8080/_broadcast (matches the Railway target URL). Default is 8090. + ANYCABLE_HTTP_BROADCAST_PORT: "8080" + ports: ["8080:8080"] + depends_on: [rails-anycable] diff --git a/cable-bench/lib/tasks/.keep b/cable-bench/lib/tasks/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench/log/.keep b/cable-bench/log/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench/public/400.html b/cable-bench/public/400.html new file mode 100644 index 0000000..640de03 --- /dev/null +++ b/cable-bench/public/400.html @@ -0,0 +1,135 @@ + + + + + + + The server cannot process the request due to a client error (400 Bad Request) + + + + + + + + + + + + + +
+
+ +
+
+

The server cannot process the request due to a client error. Please check the request and try again. If you're the application owner check the logs for more information.

+
+
+ + + + diff --git a/cable-bench/public/404.html b/cable-bench/public/404.html new file mode 100644 index 0000000..d7f0f14 --- /dev/null +++ b/cable-bench/public/404.html @@ -0,0 +1,135 @@ + + + + + + + The page you were looking for doesn't exist (404 Not found) + + + + + + + + + + + + + +
+
+ +
+
+

The page you were looking for doesn't exist. You may have mistyped the address or the page may have moved. If you're the application owner check the logs for more information.

+
+
+ + + + diff --git a/cable-bench/public/406-unsupported-browser.html b/cable-bench/public/406-unsupported-browser.html new file mode 100644 index 0000000..43d2811 --- /dev/null +++ b/cable-bench/public/406-unsupported-browser.html @@ -0,0 +1,135 @@ + + + + + + + Your browser is not supported (406 Not Acceptable) + + + + + + + + + + + + + +
+
+ +
+
+

Your browser is not supported.
Please upgrade your browser to continue.

+
+
+ + + + diff --git a/cable-bench/public/422.html b/cable-bench/public/422.html new file mode 100644 index 0000000..f12fb4a --- /dev/null +++ b/cable-bench/public/422.html @@ -0,0 +1,135 @@ + + + + + + + The change you wanted was rejected (422 Unprocessable Entity) + + + + + + + + + + + + + +
+
+ +
+
+

The change you wanted was rejected. Maybe you tried to change something you didn't have access to. If you're the application owner check the logs for more information.

+
+
+ + + + diff --git a/cable-bench/public/500.html b/cable-bench/public/500.html new file mode 100644 index 0000000..e4eb18a --- /dev/null +++ b/cable-bench/public/500.html @@ -0,0 +1,135 @@ + + + + + + + We're sorry, but something went wrong (500 Internal Server Error) + + + + + + + + + + + + + +
+
+ +
+
+

We're sorry, but something went wrong.
If you're the application owner check the logs for more information.

+
+
+ + + + diff --git a/cable-bench/public/icon.png b/cable-bench/public/icon.png new file mode 100644 index 0000000..c4c9dbf Binary files /dev/null and b/cable-bench/public/icon.png differ diff --git a/cable-bench/public/icon.svg b/cable-bench/public/icon.svg new file mode 100644 index 0000000..04b34bf --- /dev/null +++ b/cable-bench/public/icon.svg @@ -0,0 +1,3 @@ + + + diff --git a/cable-bench/public/robots.txt b/cable-bench/public/robots.txt new file mode 100644 index 0000000..c19f78a --- /dev/null +++ b/cable-bench/public/robots.txt @@ -0,0 +1 @@ +# See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file diff --git a/cable-bench/railway.toml b/cable-bench/railway.toml new file mode 100644 index 0000000..134d13f --- /dev/null +++ b/cable-bench/railway.toml @@ -0,0 +1,11 @@ +# Per-service Railway config for the Rails cable-bench targets +# (rails-solidcable / rails-actioncable / rails-anycable). +# +# The repo-root railway.toml points at backend/Dockerfile, which is wrong for +# this service. Uploading this file as the build root via +# `railway up --service --path-as-root cable-bench/` overrides the repo +# default and uses this local Dockerfile. The three services share the image +# and differ only by the BENCH_MODE env var. + +[build] +dockerfilePath = "Dockerfile" diff --git a/cable-bench/script/.keep b/cable-bench/script/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench/storage/.keep b/cable-bench/storage/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench/tmp/.keep b/cable-bench/tmp/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench/tmp/pids/.keep b/cable-bench/tmp/pids/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench/tmp/storage/.keep b/cable-bench/tmp/storage/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench/vendor/.keep b/cable-bench/vendor/.keep new file mode 100644 index 0000000..e69de29 diff --git a/cable-bench/vendor/javascript/.keep b/cable-bench/vendor/javascript/.keep new file mode 100644 index 0000000..e69de29 diff --git a/docs/rails-comparison.md b/docs/rails-comparison.md new file mode 100644 index 0000000..b2c8673 --- /dev/null +++ b/docs/rails-comparison.md @@ -0,0 +1,203 @@ +# Rails in the comparison: Action Cable vs Solid Cable vs Async::Cable vs AnyCable + +The repo behind [anycable.io/compare/rails-actioncable](https://anycable.io/compare/rails-actioncable). +Four WebSocket adapters for the same Rails app, same Railway box, same +shared-tenant window: + +- **Action Cable** on the Redis adapter (the Rails default). WebSockets + terminate in-process inside Puma's Ruby threads; Redis carries pub/sub. +- **Solid Cable** (Rails 8 default-stack adapter). Same in-process + termination, no Redis; the adapter polls a database table (default + every 100 ms) for new messages. +- **Async::Cable** ([socketry/async-cable](https://github.com/socketry/async-cable)): + serves Action Cable in-process on [Falcon](https://github.com/socketry/falcon), + a fiber-based reactor, instead of Puma's threads. Same wire protocol; a + different concurrency engine. Built on `actioncable-next`, so it runs on + stable Rails 8.1. +- **AnyCable** in full RPC mode: Rails runs a gRPC backend + (`bundle exec anycable`) for auth and commands, and `anycable-go` + (a separate Go process) holds the WebSockets and fans out broadcasts. + +The whole point of the comparison: all four are **Action Cable-compatible +at the app and client level**. The channel code, the `stream_from` calls, +and the Turbo Stream broadcasts are identical in all four. Only one line +of `config/cable.yml` and the process topology change. So any behavioral +difference is the adapter, not the app. + +## The one thing that differs on the wire + +Action Cable, Solid Cable and Async::Cable speak the base +`actioncable-v1-json` protocol, which is **at-most-once**: a broadcast that +goes out while a client is briefly offline is gone, because there is no +history to replay on reconnect. AnyCable speaks the extended +`actioncable-v1-ext-json` protocol, which keeps **per-stream history with an +epoch and offset**, so a reconnecting client resumes from the last message +it saw. The AnyCable JS client negotiates the extended protocol +automatically; a plain Action Cable client still connects over the base +protocol. That single difference drives the reliability gap below. + +Note Async::Cable: switching Puma threads for Falcon fibers changes the +runtime, not the guarantees. It is still in-process (drops on deploy) and +still speaks the base protocol (loses messages under jitter, same as the +others). Falcon does not close the reliability or deploy gap, because those +gaps come from the protocol and the topology. + +## Bench harness + +No new bench-runner endpoints. The AnyCable JS driver (`@anycable/core`) +already speaks both protocols, so the existing `bench-jitter-anycable`, +`bench-idle-anycable`, and `bench-avalanche-anycable` endpoints serve all +four Rails targets, parameterized by: + +- `cableUrl`: the `ws://…/cable` endpoint per target. +- `broadcastUrl`: the Rails `POST /_bench/broadcast` trigger + (`ActionCable.server.broadcast`), or anycable-go's `/_broadcast`. +- `channel`: `BenchmarkChannel` (real Rails channel), vs `$pubsub` for + the bare-stream nodejs targets. +- `acProtocol`: `actioncable-v1-json` for Action Cable / Solid Cable / + Async::Cable, `actioncable-v1-ext-json` for AnyCable. + +The Puma/Redis and Solid Cable targets live in `cable-bench/` (named +`cable-bench` because `rails` is a reserved Railway app name): one Docker +image, two modes via `BENCH_MODE`. AnyCable runs the same app as its gRPC +RPC backend with `anycable-go-rails` as the gateway. Async::Cable runs from +`cable-bench-falcon/`, a copy of the app booted on Falcon +(`bundle exec falcon serve`) with `actioncable-next` + `async-cable`. +Manifest entries are in `backend/src/bench/tests-manifest.ts` under each +rubric. + +### Sharding is mandatory for latency and jitter + +All numbers below are **sharded**. A single bench-runner holding thousands +of `@anycable/core` cables saturates its own Node event loop and distorts +the measurement in both directions: receive latency inflates and delivery +deflates, worst for the extended-protocol AnyCable client because it does +more per-message work. So latency, jitter, and capacity runs are sharded +across 13 drivers (~250 to 770 cables each), keeping each driver well under +the saturation point; the percentiles are merged across the union of all +samples via `mergeJitterResults`. See the bench repo README, "Running the +latency (and jitter) test." + +## Results + +All numbers from sharded runs in one shared-tenant Railway window +(2026-06-28), captured in +`backend/results/rails-sharded-2026-06-28.json`. The in-process targets ran +Puma with 8 workers × 5 threads; Async::Cable ran Falcon with 8 processes. + +### Roundtrip latency (steady network, 100% delivery) + +AnyCable is fastest at both sizes: fan-out happens in the Go gateway rather +than in Ruby, so it stays at single-digit milliseconds where the in-process +adapters climb with load. Action Cable on Puma is close behind; Async::Cable +on Falcon sits a touch higher; Solid Cable carries a fixed floor from its +100 ms database poll. + +| Adapter | 1K p50 / p99 | 5K p50 / p99 | +| --- | --- | --- | +| Solid Cable | 62 / 119 ms | 74 / 164 ms | +| Async::Cable (Falcon) | 11 / 80 ms | 20 / 71 ms | +| Action Cable (Puma) | 9 / 47 ms | 13 / 57 ms | +| AnyCable | **4 / 23 ms** | **7 / 31 ms** | + +### Delivery under jitter (5K subscribers, ~2 s drops every ~15 s) + +The strongest AnyCable win. The base protocol has no resume, so whatever +lands during an offline window is lost. All three in-process adapters behave +identically here, because the loss is in the protocol, not the runtime. + +| Adapter | Delivery | p50 | p99 (replay tail) | +| --- | --- | --- | --- | +| Solid Cable | **78.1%** | 71 ms | no resume | +| Action Cable | **78.1%** | 12 ms | no resume | +| Async::Cable (Falcon) | **78.1%** | 18 ms | no resume | +| AnyCable | **99.9%** | 7 ms | ~6.0 s | + +AnyCable's longer p99 is the resumed history landing a beat late on +reconnect. It lands; the in-process adapters drop about a fifth of +broadcasts outright. + +### Capacity: 10K under load, and idle-to-break + +In-process Action Cable has a reputation for collapsing under load. With a +properly sharded load generator, it does not at everyday sizes. + +**Under broadcast load, all four hold 10K at 100%.** AnyCable keeps the +tightest tail; Solid Cable's poll shows in its p99. + +| Adapter | 10K subs (delivery / p99) | +| --- | --- | +| Solid Cable | 100% / 200 ms | +| Action Cable | 100% / 84 ms | +| Async::Cable (Falcon) | 100% / 112 ms | +| AnyCable | **100% / 31 ms** | + +**Idle-to-break, on identical 32 GB boxes, default 8-worker config.** Ramped +idle connections until the holding box failed (raw data in +`backend/results/rails-capacity-break-2026-06-28.json`): + +| Adapter | Max held (0 fail) | Peak RAM | What capped it | +| --- | --- | --- | --- | +| Action Cable (Puma) | ~52K | 2.3 GB | 8-worker file-descriptor ceiling | +| Solid Cable (Puma) | ~52K | 2.5 GB | 8-worker file-descriptor ceiling | +| Async::Cable (Falcon) | ~97K | 27 GB | memory, ~290 KB/conn | +| AnyCable (Go gateway) | **600K+** | 27 GB | not broken; load fleet maxed out | + +Three findings. The Puma adapters wall at ~52K using only ~2.5 GB: an +8-worker file-descriptor ceiling, not memory (raising worker fd limits +lifts it). Async::Cable on Falcon is memory-bound at ~97K, because each +fiber-backed connection costs roughly 290 KB, about six times the others. +AnyCable held 600K idle connections with zero failures across a 50-runner +fleet before we ran out of load generators; its gateway memory scaled +linearly to 27 GB (~47 KB/conn, the same per-connection cost as Puma), +about 84% of the box, so its real ceiling is near ~700K and memory-bound. +The per-connection RAM is similar to Puma's; what differs is that one Go +process has no per-worker fd wall, so it uses the whole box instead of +stalling at 52K with 90% of the box idle. + +Finding a gateway's true ceiling takes roughly one load driver per 10K +connections: a single Node driver tops out around 10K cables, so reaching +600K needed the full 50-runner fleet. With fewer drivers the fleet caps the +result before the server does, so capacity-to-break runs scale the driver +count to the target. + +### Deploy survival (avalanche, 5K clients, real app redeploy) + +In-process WebSocket servers drop every connection when the app restarts, +Puma and Falcon alike. On our test the in-process adapters went fully dark +on the redeploy and stayed down for roughly 7.5 to 8 seconds before about +96% of clients reconnected. For AnyCable the redeploy hit the Rails gRPC +backend, not the gateway holding the sockets, so its connections were never +dropped: zero seconds of downtime. + +The "Down for" column below is how long connections stayed dropped after the +redeploy before the reconnect storm settled. + +| Adapter | Dropped | Down for | Reconnected | +| --- | --- | --- | --- | +| Action Cable | all 5,000 | 7.5 s | 96.3% (187 still out at cutoff) | +| Solid Cable | all 5,000 | 7.6 s | 95.7% (215 still out at cutoff) | +| Async::Cable (Falcon) | all 5,000 | 8.0 s | 96.4% (179 still out at cutoff) | +| AnyCable | **0** | **0 s** | n/a | + +The AnyCable run redeployed its Rails RPC backend the same way; the gateway +reported no disconnect across the full 180 s observation window, so the +downtime is zero by construction. + +## The takeaway + +On Rails, AnyCable leads on latency at every scale (7 ms p50 at 5K vs 13 ms +for Action Cable, 20 ms for Async::Cable, 74 ms for Solid Cable), because +fan-out runs in Go off the Ruby process. It wins everything that decides +whether realtime holds up in production: 100% delivery under jitter where +the base protocol drops about a fifth of broadcasts, and every connection +kept alive across an app deploy where the in-process adapters drop all of +them. Capacity ties at everyday sizes (all four hold 10K at 100%) but splits +sharply past that: AnyCable held 600K idle connections where the Puma +adapters wall at ~52K on a file-descriptor ceiling and Falcon runs out of +memory at ~97K. Async::Cable on Falcon is a real alternative runtime to +Puma with latency and capacity in the same range, but it shares the +in-process limits: at-most-once and deploy-fragile. Solid Cable's own edge +is operational: no Redis, just the database, at the cost of a fixed +polling-latency floor. All four keep your channels and Turbo Streams exactly +as written. diff --git a/docs/railway-ops.md b/docs/railway-ops.md index 5e8bd53..43b3896 100644 --- a/docs/railway-ops.md +++ b/docs/railway-ops.md @@ -94,16 +94,27 @@ curl -s -X POST https://backboard.railway.com/graphql/v2 \ "variables":{"input":{"serviceId":"","environmentId":"","memoryGB":0.5,"vCPUs":1}}}' ``` -Or stop a service entirely: +Or stop a service entirely. Two working ways: ```bash -# Stop a deployed container without deleting the service +# CLI (idempotent; "No deployments found" when already down) +railway down --service --yes + +# GraphQL: remove the latest deployment (halts billing, keeps the service shell) curl -s -X POST https://backboard.railway.com/graphql/v2 \ -H "Authorization: Bearer $RAILWAY_TOKEN" -H "Content-Type: application/json" \ - -d '{"query":"mutation S($id: String!) { deploymentStop(id: $id) }", + -d '{"query":"mutation S($id: String!) { deploymentRemove(id: $id) }", "variables":{"id":""}}' ``` +Do NOT use the `deploymentStop` mutation: it returns success and the +container keeps running (verified the hard way). After teardown, verify +externally — curl each public domain expecting 404/000 — and remember that +a limits change after a stop can respawn a fresh deployment, and that +deployment status `SUCCESS` is a build record which persists after a stop. +The `fleet-watchdog` GitHub Action (scripts/fleet-watchdog.mjs) opens an +issue if anything is left running between campaigns. + For the bench-runner shards specifically: they're ~64 MB each at idle, so 50 of them is only ~3 GB total. The bigger savings come from the high-RAM target services (`uws-server`, `anycable-go`, `socketio-server` diff --git a/fleet-manifest.json b/fleet-manifest.json new file mode 100644 index 0000000..a00becb --- /dev/null +++ b/fleet-manifest.json @@ -0,0 +1,72 @@ +{ + "$comment": "Fleet-as-code: what the Railway project is supposed to look like. bench:preflight and bench:fleet diff live state against this. Update it when the fleet's intended shape changes; drift between this file and reality is a finding, whichever side is wrong.", + "project": "gentle-commitment", + "projectId": "fd842a43-8d78-48c0-879f-4b5311c8c004", + "environment": "production", + "environmentId": "284c9b0a-57cb-4281-a4bd-868473a94a47", + "runnerPattern": "^bench-runner(-\\d+)?$", + "$runnerNote": "bench-runner-81 runs the Centrifugo target image (repurposed 2026-07; the project is at the 100-service cap). Never use it as a load-generator shard.", + "excludedRunners": ["bench-runner-81"], + "sharedSecrets": ["BENCH_RUNNER_TOKEN", "ANYCABLE_BROADCAST_SECRET"], + "idleExpectation": "torn down", + "$idleNote": "Between campaigns every service should have NO active deployment (teardown = deploymentRemove per service). Any live deployment outside a run window is billing for nothing; the fleet watchdog alerts on it.", + "targets": { + "anycable-go": { + "role": "AnyCable OSS gateway", + "expectEnv": [], + "forbidEnv": { "ANYCABLE_BROADCAST_ADAPTER": "nats" }, + "notes": "http broadcast adapter is the default and the only one the bench publish path reaches; an explicit nats setting silently delivers nothing (200 OK, 0% delivery)." + }, + "anycable-go-pro": { + "role": "AnyCable Pro gateway", + "expectEnv": [], + "forbidEnv": { "ANYCABLE_BROADCAST_ADAPTER": "nats" }, + "notes": "Custom Dockerfile: deploy only with per-service railway.toml + --path-as-root anycable-pro/. Needs GITHUB_TOKEN build arg. EMBED_NATS adds a broadcast hop; state it next to throughput numbers." + }, + "socketio-server": { + "role": "Socket.io (default)", + "expectEnv": [], + "notes": "SOCKETIO_CSR must be UNSET here; check NODE_OPTIONS heap fits the container after any resize." + }, + "socketio-server-csr": { + "role": "Socket.io with Connection state recovery", + "expectEnv": ["SOCKETIO_CSR"], + "notes": "SOCKETIO_CSR=1 at boot. Internal DNS may still resolve the pre-rename hostname; see tests-manifest." + }, + "uws-server": { + "role": "uWebSockets.js", + "expectEnv": [], + "notes": "" + }, + "rails-actioncable": { + "role": "Rails + Puma Action Cable", + "expectEnv": ["WEB_CONCURRENCY", "RAILS_MAX_THREADS"], + "notes": "WEB_CONCURRENCY only works because puma.rb has an explicit workers directive (stock puma.rb ignores it). Verify worker count from boot logs, never from env alone. Reset to 8 after any experiment." + }, + "rails-solidcable": { + "role": "Rails + Solid Cable", + "expectEnv": ["WEB_CONCURRENCY", "RAILS_MAX_THREADS"], + "notes": "Same puma.rb caveat as rails-actioncable." + }, + "rails-asynccable": { + "role": "Falcon + Async::Cable", + "expectEnv": [], + "notes": "falcon serve --count must match Puma's worker count. Needs the raw_transmit shim until async-cable ships it." + }, + "rails-anycable": { + "role": "Rails RPC backend for AnyCable", + "expectEnv": ["ANYCABLE_SECRET"], + "notes": "Bring this up BEFORE anycable-go-rails; the gateway caches a dead grpc connection otherwise. DB pool must be >= RPC pool (30) for storm tests." + }, + "anycable-go-rails": { + "role": "AnyCable gateway for the Rails comparison", + "expectEnv": [], + "notes": "Image service (no repo build): railway down removes its deployment and railway redeploy then fails; revive with a variable nudge (railway variables --set NUDGE=)." + }, + "Redis": { + "role": "Broker for Action Cable / Socket.io Redis adapter", + "expectEnv": [], + "notes": "DB service with a volume: cannot be revived via CLI, dashboard only. After it returns, redeploy dependent Rails targets (they cache dead connections)." + } + } +} diff --git a/scripts/fleet-watchdog.mjs b/scripts/fleet-watchdog.mjs new file mode 100644 index 0000000..0b0ceb3 --- /dev/null +++ b/scripts/fleet-watchdog.mjs @@ -0,0 +1,152 @@ +// Fleet teardown watchdog. Runs on a GitHub Actions cron (see +// .github/workflows/fleet-watchdog.yml) and checks whether any Railway +// service in the bench project still has a live deployment. Railway bills +// per-minute for allocated resources and the fleet is supposed to be torn +// down after every campaign; a forgotten fleet burns money silently. +// +// Behavior: +// - live services found → open (or update) an issue labeled fleet-watchdog +// listing them, so somebody tears the fleet down or acknowledges a +// campaign in progress by keeping the issue open. +// - nothing live → close any open fleet-watchdog issue. +// +// Env (set in the workflow): +// RAILWAY_TOKEN account token (create at railway.com/account/tokens; +// CLI login tokens expire, account tokens persist) +// PROJECT_ID Railway project uuid +// ENVIRONMENT_ID Railway environment uuid +// GITHUB_TOKEN provided by Actions (needs issues: write) +// GITHUB_REPOSITORY owner/repo, provided by Actions +// +// No dependencies: plain fetch, runs on any Node >= 20. + +const { + RAILWAY_TOKEN, + PROJECT_ID, + ENVIRONMENT_ID, + GITHUB_TOKEN, + GITHUB_REPOSITORY, +} = process.env; + +for (const [name, v] of Object.entries({ + RAILWAY_TOKEN, + PROJECT_ID, + ENVIRONMENT_ID, + GITHUB_TOKEN, + GITHUB_REPOSITORY, +})) { + if (!v) { + console.error(`Missing env var: ${name}`); + process.exit(1); + } +} + +const LIVE = new Set(["SUCCESS", "DEPLOYING", "BUILDING", "INITIALIZING", "WAITING"]); +const LABEL = "fleet-watchdog"; + +async function railway(query, variables) { + const res = await fetch("https://backboard.railway.com/graphql/v2", { + method: "POST", + headers: { + Authorization: `Bearer ${RAILWAY_TOKEN}`, + "Content-Type": "application/json", + "User-Agent": "curl/8.4.0", + "Accept-Encoding": "identity", + }, + body: JSON.stringify({ query, variables }), + }); + const json = await res.json(); + if (json.errors) { + throw new Error(`Railway GraphQL error: ${JSON.stringify(json.errors)}`); + } + return json.data; +} + +async function github(method, path, body) { + const res = await fetch(`https://api.github.com${path}`, { + method, + headers: { + Authorization: `Bearer ${GITHUB_TOKEN}`, + Accept: "application/vnd.github+json", + "User-Agent": "fleet-watchdog", + }, + body: body ? JSON.stringify(body) : undefined, + }); + if (!res.ok) { + throw new Error(`GitHub ${method} ${path}: HTTP ${res.status} ${await res.text()}`); + } + return res.status === 204 ? null : res.json(); +} + +const data = await railway( + `query Env($id: String!) { + environment(id: $id) { + serviceInstances { + edges { node { serviceName latestDeployment { status createdAt } } } + } + } + }`, + { id: ENVIRONMENT_ID }, +); + +const services = data.environment.serviceInstances.edges.map((e) => e.node); +const live = services.filter( + (s) => s.latestDeployment && LIVE.has(s.latestDeployment.status), +); + +console.log(`${services.length} services, ${live.length} live`); +for (const s of live) { + console.log(` live: ${s.serviceName} (${s.latestDeployment.status}, since ${s.latestDeployment.createdAt})`); +} + +const issues = await github( + "GET", + `/repos/${GITHUB_REPOSITORY}/issues?state=open&labels=${LABEL}`, +); +const existing = issues[0]; + +if (live.length === 0) { + if (existing) { + await github("POST", `/repos/${GITHUB_REPOSITORY}/issues/${existing.number}/comments`, { + body: "Fleet is fully torn down; closing.", + }); + await github("PATCH", `/repos/${GITHUB_REPOSITORY}/issues/${existing.number}`, { + state: "closed", + }); + console.log(`Closed issue #${existing.number}`); + } else { + console.log("Fleet down, no open issue. Nothing to do."); + } + process.exit(0); +} + +const oldest = live + .map((s) => new Date(s.latestDeployment.createdAt)) + .sort((a, b) => a - b)[0]; +const hoursUp = ((Date.now() - oldest.getTime()) / 3.6e6).toFixed(1); + +const body = [ + `**${live.length} Railway service(s) are live and billing** (oldest deployment up ~${hoursUp}h).`, + "", + "If a benchmark campaign is running, keep this issue open as the reminder and close it after teardown. Otherwise: tear the fleet down (stop deployments per service; limits changes alone keep billing) and verify public domains return 404.", + "", + "| Service | Status | Deployed |", + "|---|---|---|", + ...live.map( + (s) => `| ${s.serviceName} | ${s.latestDeployment.status} | ${s.latestDeployment.createdAt} |`, + ), + "", + `_Updated ${new Date().toISOString()} by the fleet watchdog._`, +].join("\n"); + +if (existing) { + await github("PATCH", `/repos/${GITHUB_REPOSITORY}/issues/${existing.number}`, { body }); + console.log(`Updated issue #${existing.number}`); +} else { + const issue = await github("POST", `/repos/${GITHUB_REPOSITORY}/issues`, { + title: "Bench fleet is live — tear down or acknowledge", + body, + labels: [LABEL], + }); + console.log(`Opened issue #${issue.number}`); +}