land #8783: deopt exact callbacks and captured updates #3693

Workflow file for this run

.github/workflows/gc-ratchet.yml at f5e1698

	name: GC Ratchet

	# Regression gate for GC retention, evacuation accounting, and memory against
	# the pinned baseline in benchmarks/gc_ratchet/baseline/gc-ratchet-v1.json.
	#
	# THIS JOB IS DESIGNED TO BE ABLE TO FAIL. There is no `continue-on-error`, no
	# `\|\| true`, and no pipe between the checker and the shell's exit status. That
	# is deliberate and load-bearing: `gc-stress` in test.yml is
	# `continue-on-error: true`, which is why a regression sat behind it through
	# three merges. A gate that cannot fail is not a gate.
	#
	# NOT the public benchmark baseline. benchmarks/run_public_baseline.sh produces
	# benchmarks/results/public-node-bun-v1.json (published Node/Bun evidence, gates
	# `lint`). This is the internal Perry-vs-Perry GC ratchet. Do not conflate them.

	on:
	pull_request:
	# PR arm is OPT-IN via the `run-extended-tests` label (see the header of
	# test.yml and docs/src/testing/ci-tiers.md): an unlabelled PR still gets
	# a run, but every job in it is skipped, which costs no runner slot. The
	# main-line arm (schedule / tags) is unchanged. `labeled` re-fires the run
	# when the label lands.
	types: [opened, synchronize, reopened, labeled]
	# POST-MERGE ARM: a staggered six-hourly sweep of `main`, NOT one run per merge.
	#
	# `push: branches: [main]` is what took this gate dark for two days (#7856).
	# Fourteen workflows fired on every merge, ~29 jobs each time, against a repo
	# that runs ~9 jobs concurrently; at 58 merges/day demand outran drain and the
	# queue grew without bound. On 2026-08-11 this workflow had 22 of its 25 most
	# recent `main` runs sitting `queued` (oldest 9h18m) and its last successful
	# `main` run was 2026-08-09 -- two days and five collector-touching merges
	# earlier (#7799, #7809, #7812, #7834, #7839). #7843's seven genuinely-red rows
	# landed inside that blind window.
	#
	# This is the THIRD variant of CLAUDE.md's "four ways a gate can be unable to
	# fail": not `continue-on-error`, not missing from required contexts, not
	# cancelled -- STARVED. A gate that runs four times a day and COMPLETES is
	# worth more than one that runs on every merge and never does.
	#
	# It is NOT the `concurrency:` block below (already correct, twice-repaired --
	# see its comment) and it is NOT macOS capacity: at the moment of measurement
	# the queue held 45 `ubuntu-latest` jobs against 14 `macos-14`. Rebalancing
	# pools cannot help; only cutting total job demand can.
	#
	# *DO NOT RESTORE `push: branches: [main]` HERE.* Full measurement, the two
	# #7856 claims that did not survive checking, and the cost this trades away:
	# docs/src/testing/ci-gate-scheduling.md
	schedule:
	- cron: "7 /6 * *"
	push:
	# Releases stay individually gated; tags are rare, so this costs nothing.
	tags: ["v*"]
	workflow_dispatch:

	permissions:
	contents: read

	concurrency:
	# ONE GROUP PER MAIN COMMIT, and cancel ONLY pull-request runs.
	#
	# A shared group with unconditional cancel-in-progress looked right and was
	# wrong: `main` is busy and the macOS runner pool is deep enough that a run
	# can sit queued for 40+ minutes, so every new merge cancelled the previous
	# main run before it ever got a runner. Observed on the first day this
	# workflow existed — three consecutive main runs cancelled, zero executed.
	# A gate that is always cancelled never fails, which is the same hole as
	# `continue-on-error: true` wearing a different hat.
	#
	# *SCOPING `cancel-in-progress` TO PULL REQUESTS DID NOT FIX IT (#7205).*
	# It protects a run that has already STARTED. GitHub additionally allows at
	# most one PENDING run per group: when a new run enters a group, "any
	# previously pending job or workflow in the concurrency group will be
	# cancelled" — regardless of `cancel-in-progress`. Every `main` run here is
	# pending for tens of minutes, so a merge cadence faster than the queue depth
	# kept cancelling all of them. Re-measured after the first fix: runs
	# 30707891646 (#7192), 30708370282 (#7198) and 30707189162 (#7195) all ended
	# `cancelled` with `jobs: []` — never reached a runner, zero executions in
	# three merges.
	#
	# Keying the group on the COMMIT for push events is what actually fixes it:
	# each merged commit gets a group of its own, so no two `main` runs can ever
	# contend and none can supersede another. They queue independently and each
	# eventually gets its own answer. PR runs keep sharing a per-ref group and
	# keep superseding themselves, which is still correct — only the head
	# commit's result gates the merge.
	# *#7966: KEY EVERY MAIN-LINE RUN ON `github.run_id`, NOT `github.sha`.*
	# The previous expression read `github.event_name == 'push' && github.sha \|\|
	# github.ref`. That was #7205's fix and it keyed on the event being `push` --
	# correct while the main-line arm WAS `push: branches: [main]`. #7856 moved the
	# main-line arm to `schedule:`, which falls through to `github.ref` (constant
	# `refs/heads/main`), so every scheduled run shared one group again and #7205
	# came straight back. Measured 2026-08-12 on all ten scheduled gates, the same
	# shape every time: oldest run `queued` holding the group, the two after it
	# `cancelled` with `jobs: 0`, newest `pending`. `github.run_id` is unique per
	# run, so schedule / tag-push / workflow_dispatch each get a group of their own
	# and none can supersede another. PR runs keep the shared per-ref group and
	# keep superseding themselves, which is still what we want.
	group: gc-ratchet-${{ github.event_name }}-${{ github.event_name == 'pull_request' && github.ref \|\| github.run_id }}
	cancel-in-progress: ${{ github.event_name == 'pull_request' }}

	env:
	CARGO_TERM_COLOR: always
	MACOSX_DEPLOYMENT_TARGET: "13.0"

	jobs:
	gc-ratchet:
	# PR arm is opt-in (label `run-extended-tests`); see the `on:` block.
	if: github.event_name != 'pull_request' \|\| contains(github.event.pull_request.labels.*.name, 'run-extended-tests')
	# macos-14 is arm64, which matches the `darwin-arm64` platform key the
	# baseline was captured under. The checker refuses a platform mismatch
	# rather than comparing numbers that are not comparable, so moving this to
	# a linux runner would turn the gate red, not silently weaken it.
	runs-on: macos-14
	timeout-minutes: 90
	permissions:
	contents: read
	# Read-only listing of the PR's changed files, to decide whether this
	# change can touch the collector. Same approach `changeset-gate` uses, and
	# it avoids a fetch-depth: 0 clone of a large history.
	pull-requests: read
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
	with:
	persist-credentials: false

	# Fast structural failure first: a broken harness or a tampered artifact
	# should not cost a 20-minute compiler build to discover.
	#
	# `--scope structural` is load-bearing and is the #7554 repair. This step
	# runs BEFORE the measurement step, so anything it fails on costs the
	# entire run's coverage — fourteen probes that never execute. That price
	# is correct for "this artifact is unreadable or tampered with" and badly
	# wrong for "one of its 156 cells is not bit-identical", which is what
	# actually happened: one cell's 6,768-byte spread failed this step and the
	# ratchet measured NOTHING on any branch for three days, while two GC
	# pacing changes (#7594, #7596) merged with hand-run A/Bs in its place.
	#
	# Under `structural` a per-cell defect is printed loudly and deferred; the
	# `check` step below re-derives the same defect list and fails on it, after
	# the probes have run and with the full table attached. So this cannot
	# wave a defect through to a green job — `check` is where the verdict is,
	# and tests/test_gc_ratchet.py's
	# `test_structural_preflight_defers_every_defect_it_waves_through` asserts
	# that coupling one planted defect shape at a time.
	- name: Harness unit tests and artifact validation
	run: \|
	python3 -m unittest discover -s tests -p 'test_gc_ratchet.py' -v
	python3 benchmarks/gc_ratchet/gc_ratchet.py validate --scope structural

	- name: Decide whether this change can affect the collector
	id: relevance
	env:
	GH_TOKEN: ${{ github.token }}
	EVENT_NAME: ${{ github.event_name }}
	REPOSITORY: ${{ github.repository }}
	PR_NUMBER: ${{ github.event.pull_request.number }}
	run: \|
	set -euo pipefail
	if [[ "$EVENT_NAME" != "pull_request" ]]; then
	echo "run=true" >> "$GITHUB_OUTPUT"
	echo "Not a pull request; measuring."
	exit 0
	fi
	gh api "repos/$REPOSITORY/pulls/$PR_NUMBER/files" --paginate --jq '.[].filename' > changed.txt
	# Anything under crates/ can move the collector: a codegen change
	# alters what is allocated, a transform change alters what escapes,
	# and a dependency bump can change allocator behaviour. The filter is
	# deliberately broad; it exists only to spare docs-only PRs a build.
	# If the listing is empty or the API failed, `set -e` already aborted
	# — the job does not silently fall through to "not relevant".
	if grep -qE '^(crates/\|benchmarks/gc_ratchet/\|Cargo\.(toml\|lock)$\|\.github/workflows/gc-ratchet\.yml$\|tests/test_gc_ratchet\.py$)' changed.txt; then
	echo "run=true" >> "$GITHUB_OUTPUT"
	echo "Change touches collector-relevant paths; measuring."
	else
	echo "run=false" >> "$GITHUB_OUTPUT"
	echo "No collector-relevant paths changed; the pinned baseline still holds."
	fi

	- name: Install Rust toolchain
	if: steps.relevance.outputs.run == 'true'
	run: rustup toolchain install nightly-2026-08-20 --profile minimal
	- uses: ./.github/actions/setup-llvm22

	- name: Cache cargo
	if: steps.relevance.outputs.run == 'true'
	uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6
	with:
	path: \|
	~/.cargo/registry
	~/.cargo/git
	target
	key: ${{ runner.os }}-cargo-bench-${{ hashFiles('**/Cargo.lock') }}
	restore-keys: ${{ runner.os }}-cargo-

	- name: Setup Node oracle
	if: steps.relevance.outputs.run == 'true'
	uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
	with:
	# Single source of truth: .node-version at the repo root. Node is a
	# correctness input here, not a peer benchmark: every probe's stdout
	# is diffed against it, because a probe that silently stops
	# allocating exits 0 and reports a beautifully small retained heap.
	node-version-file: .node-version

	- name: Build perry and the runtime archives
	if: steps.relevance.outputs.run == 'true'
	run: \|
	set -euo pipefail
	# perry-runtime and perry-stdlib are rlib-only. libperry_runtime.a and
	# libperry_stdlib.a come from the -static wrapper crates; building
	# without them links a stale archive and makes the whole measurement
	# vacuous. The package set is fixed here so cargo feature unification
	# is identical to the set used to capture the baseline.
	cargo build --release -p perry -p perry-runtime-static -p perry-stdlib-static
	for artifact in perry libperry_runtime.a libperry_stdlib.a; do
	test -s "target/release/$artifact" \
	\|\| { echo "::error::target/release/$artifact was not produced"; exit 1; }
	done

	- name: Verify the Node oracle version
	if: steps.relevance.outputs.run == 'true'
	run: \|
	set -euo pipefail
	expected="v$(tr -d 'v \n' < .node-version)"
	actual="$(node --version)"
	test "$expected" = "$actual" \
	\|\| { echo "::error::expected Node $expected, found $actual"; exit 1; }

	- name: Measure
	if: steps.relevance.outputs.run == 'true'
	env:
	PERRY_RUNTIME_DIR: ${{ github.workspace }}/target/release
	# Deterministic link: use the prebuilt full stdlib rather than letting
	# the auto-optimizer rebuild a workload-specific archive mid-run.
	PERRY_NO_AUTO_OPTIMIZE: "1"
	run: \|
	set -euo pipefail
	mkdir -p .bench-results
	# Load is recorded inside the measurement JSON too; printing it here
	# keeps it in the log next to the numbers.
	uptime
	# Pass the oracle explicitly rather than letting the harness search.
	# A run that cannot reach the oracle now fails the gate — "we did not
	# verify" must not look like "verified" — so resolution must not be
	# left to chance.
	python3 benchmarks/gc_ratchet/gc_ratchet.py measure \
	--perry target/release/perry \
	--repeats 7 \
	--node "$(command -v node)" \
	--output .bench-results/gc-ratchet-current.json

	# No pipe, no `\|\| true`, no continue-on-error: this step's exit status is
	# the gate. The `shared_ci` profile gates retention and the evacuation
	# counters, which are semantic and transfer across machine classes, and
	# explicitly does not gate RSS or wall time, which do not. See
	# benchmarks/gc_ratchet/tolerances.json for every band and its reasoning.
	- name: Check against the pinned baseline
	if: steps.relevance.outputs.run == 'true'
	run: \|
	python3 benchmarks/gc_ratchet/gc_ratchet.py check \
	--current .bench-results/gc-ratchet-current.json \
	--profile shared_ci

	- name: Upload measurement
	if: always() && steps.relevance.outputs.run == 'true'
	uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
	with:
	name: gc-ratchet-${{ github.sha }}
	path: \|
	.bench-results/gc-ratchet-current.json
	benchmarks/gc_ratchet/baseline/gc-ratchet-v1.json
	retention-days: 90

Navigation Menu

Uh oh!

Uh oh!

land #8783: deopt exact callbacks and captured updates #3693

Workflow file

land #8783: deopt exact callbacks and captured updates #3693

Uh oh!

Workflow file for this run