diag(gc): report incremental cycles that start and never finish (#7909) - #7923
Conversation
9cfd0c3 to
1f21946
Compare
|
Warning Review limit reached
Next review available in: 36 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
A budgeted incremental cycle emits nothing until it COMPLETES — the `[gc]`
trace is written by `gc_finish_budgeted_cycle`. A cycle that is started and
then starved is therefore completely invisible: no trace line, no counter,
nothing, while the mutator pays the SATB mark barrier on every heap store,
every shadow-slot root store and every allocation for as long as it stays
open.
`gc-handoff/apps/asyncpipe.ts` is in exactly that state, which is why it reads
as "zero GC cycles, but a third of the leaf profile is collector machinery".
`PERRY_GC_DIAG=1` (existing knob, no new knob) now prints at the process-exit
boundary:
[gc-incremental] cycle_starts=1 steps=15 completions=0 active_at_exit=true
mark_barrier_arms=1 mark_barrier_armed_us=37214
skips(reentrant=0 no_trigger=2 start_blocked=0 resume_blocked=0)
safepoints_blocked_by_budgeted=0 copying_minors=0
loop_polls=1 poll_arm_events=0 poll_armed_at_exit=0
One cycle started, fifteen steps, zero completions, still active at exit, the
mark barrier armed for 37 ms of a 127 ms program, and not one collection.
No behaviour change. The mechanism, and the measured negative on the obvious
fix (+51.4 % instructions, +57 % RSS, because the minors it unblocks are priced
by #7915), are written up in `gc-handoff/GC7909-NOTES.md`.
Claude-Session: https://claude.ai/code/session_012B8z92S82sCfqCrVqrFgS2
1f21946 to
f25829f
Compare
1 participant
Closes the investigation half of #7909. No behaviour change — this ships the
instrument, the mechanism, and a measured negative on the obvious fix.
The question #7909 asked
Answer
The heap does trigger a collection.
gc_budgeted_due_trigger()returnsSome(ArenaBytes)from the young-generation scavenge cap(
young_scavenge_cap_due, 16 MB) and — because nothing ever collects — it staysdue for the rest of the run. What never happens is the collection.
gc_runtime_safepoint()(
promise/microtasks.rs), which starts a budgeted incremental cycle as soonas any trigger is due.
low_pause_non_movingbyconstruction (
gc_start_budgeted_minor_fallback_cycle_with_snapshotsetsevacuation_policy_allowed = !low_pause_non_moving), so it cannot evacuateand cannot lower
copying_from_space_in_use_bytes()— the quantity the captests.
gc_safepoint_moving_minorrejects every precisesafepoint at its
budgetedentry guard, so the collector that could clearthe trigger never runs again.
program — the cycle cannot finish.
The result is a self-sustaining stall, and it is completely silent: the
[gc]trace is written bygc_finish_budgeted_cycle, so a cycle that nevercompletes emits nothing at all.
The alloc-point path already routes nursery pressure away from the budgeted
stepper for exactly this reason (
gc/policy.rs, thegc_moving_loop_polls_enabled()defer arm). The host-safepoint path does not. That asymmetry is the defect.
The loop poll is not involved
PERRY_GC_MOVING_LOOP_POLLS=0acts here only throughnursery_cap_active(),which is
gc_moving_loop_polls_enabled()(policy.rs). The new counterssay so directly:
poll_arm_events=0,loop_polls=1— the back-edge poll isarmed zero times and taken once (the startup seed release) in the whole program.
PERRY_GC_SCAVENGE_NURSERY_MB=4096, which moves the cap and nothing else,reproduces the knob's effect to three digits.
PERRY_GC_SCAVENGE_NURSERY_MB=4096PERRY_GC_MOVING_LOOP_POLLS=0PERRY_GC_SCAVENGE=0What this PR adds
PERRY_GC_DIAG=1(existing knob, no new knob) now prints one line at theprocess-exit boundary:
safepoints_blocked_by_budgeted=5is the lockout, measured: five precisesafepoints rejected by a cycle that never completes and never collects.
mark_barrier_armed_usis the cost number: the SATB mark barrier armed for37 ms of a 127 ms program, taxing every heap store, every shadow-slot root
store and every allocation for a cycle that collects nothing. It also explains
why
PERRY_WRITE_BARRIERS=0was +0.9 % — that knob disables the generationalcodegen store barrier; this is the incremental mark barrier, armed separately.
The obvious fix: tried, measured, NOT shipped
Giving the precise collector first refusal at the outermost pump boundary (run
gc_safepoint_moving_minor()beforegc_runtime_safepoint()) removes the stallexactly as designed —
cycle_starts1 → 0,mark_barrier_armed_us37 214 → 0,copying_minors0 → 2, all 19 corpus programs byte-identical — and costs:The control rows are inert to three digits, so the +51 % is the price of the two
minors it unblocks, not of the change. Those minors are priced by #7915: one is
a 134 ms minor that copied zero objects and zero bytes, spending its time in
runtime_mutable_scannersover 218 455 pointer roots across 82 registeredscanners. On
asyncpipe, collecting is worse on every axis.#7909 cannot be closed before #7915. They are one chain: #7909 is the
mutator paying for a collection that is scheduled and then neither performed nor
cancelled; #7915 is that performing it costs more than not performing it. The
reorder belongs after #7915, and is written up in
gc-handoff/GC7909-NOTES.md§4so it can be picked up as-is.
Tests
an_active_budgeted_cycle_locks_out_the_moving_minor_and_keeps_the_barrier_armedpins the composition and asserts its own subject was live at every step — the
arena trigger really is due before the call; the call under test really did start
a cycle (
incremental_cycle_starts+1); the moving-minor rejection is attributedto the
budgetedguard specifically rather than to a transient one; the barrierreally was armed; and the completion counter moves too, so
starts > completionscan be read as a stall rather than as a dead fixture.
arm_events_count_arms_and_the_word_is_reportedpins the poll-arming pair whosezero retired the loop-poll hypothesis.
Validation
cargo test --release -p perry-runtime --lib(RUST_TEST_THREADS=1):2180 passed, 0 failed, 4 ignored
gc-handoffcorpus programs byte-identical, exit 0;asyncpipe,iso_miss,interp,pipelineandshapesalso byte-identical and exit 0under
PERRY_GC_PROTECT_FROMSPACE=1 PERRY_GC_PROTECT_FROMSPACE_DEPTH=800andunder
PERRY_GC_VERIFY_EVACUATION=1PERRY_GC_DIAG; every countersits on a cold path (cycle start/complete, barrier arm/disarm, a blocked
safepoint, a step that did no work)
cargo fmt --all -- --check,scripts/check_file_size.sh