Skip to content

Commit 6aab3ff

Browse files
astubbsclaude
andauthored
tooling(inflight): a vetting worklist, the grooming procedure, and the first sweep of the notes for v6 (#476)
The road to v6 needs the in-flight notes vetted - which are still real, and in what order to read them - and nothing recorded when a note was last confirmed against the tree. git log could not say: the package rename of 2026-08-26/27 rewrote every file, so last-touched was the same day for nearly all of them, and the only way to know which of the open notes were still true was to read all of them. Tooling (bin/inflight.mjs): - `vet`: every open note on the baseline (or --ref a branch), partitioned on a new `<!-- inflight-vetted: YYYY-MM-DD - what was checked -->` marker, ordered by the index's group order and then oldest first by first-added date, each annotated with the cheap staleness signals a script can see - every cited fork number settled, the filename's number settled, a cited path or symbol that no longer resolves, a stated delete-when condition. A signal is a reason to open the note, never a verdict; the tool closes nothing. --area splits a sweep between agents. - `rank` orders the rows inside each impact bucket oldest first rather than alphabetically (operator ruling on finding the buckets were in filename order). The session-start index keeps its path order. - firstAddedDates in bin/lib/git.mjs, one git log for every note's first-added date; symbolsPresent tokenises the source in-process (112ms against 4.8s for the git grep it replaced); numbersByValue caches issue and PR states for a day, with the kind, since the two share a counter. - bin/check-inflight-tags.sh validates the marker's shape and names a `>` inside it as the cause. Self-tests for the gate and eleven negative-controlled checks in bin/test-inflight.mjs. Procedure: docs/grooming.md owns the sweep - when to run one, how it is split by area, the dispatch prompt, what comes back and where it is consolidated, and the traps the first sweep met. Routed from the root AGENTS.md table, the "Vetting a note" section of docs/inflight/AGENTS.md (the per-note contract: the marker, the five outcomes, the owner gate), the release procedure, and vet's own output. The sweep: six agents, one per area. Every open note on master now carries a stamp or a state. Owner-gated notes - a bug at stall or worse, the release record, a public-API contract - were not changed by the sweep; each carried a PROPOSED marker with the evidence, consolidated and ranked in process-candidate-ranking.md. The owner's pass on 2026-09-08 accepted every proposal: three closes, one merge (bug-wedged-after-poisoned-transaction into its sibling), one retype (bug-857-family is a register), and the shrinks and corrections, applied by three further agents; #139 was ruled out of v6 scope and its note deferred with its false volatility premise corrected. The register keeps the agents' reading of what gates v6, stated with their confidence. Overtaken by master mid-sweep: #466 landed the revoke-path drain fix, deleting one note the sweep had read as a v6 gate and removing the spin another described; both are re-premised here. Two out-of-area passages the sweep proved stale are corrected in place - the ruleset removals in docs/ci.md and the Jackson comment in .github/dependabot.yml - and docs/refactoring.md drops an entry describing a defect the tree no longer has. Rejected: a numeric priority field. Impact is the coarse order, a second axis would compete with it and rot the way the next- prefix did, and the v6 cut is the deferred state. Co-authored-by: Claude Fable 5.1 (1M context) <noreply@anthropic.com>
1 parent 745b1f6 commit 6aab3ff

186 files changed

Lines changed: 2148 additions & 1213 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/dependabot.yml

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -51,29 +51,31 @@ updates:
5151
- "version-update:semver-minor"
5252
- "version-update:semver-major"
5353
# The second Jackson pin: the jackson-bom import in
54-
# parallel-consumer-example-streams, which holds that module at 2.18.9. It needs
54+
# parallel-consumer-example-streams, which holds that module on the 2.18 line
55+
# (jackson.examples.version in the examples parent says which patch). It needs
5556
# the same hand management as the databind pin above, and it is a *different*
5657
# coordinate, so the rule above does not cover it.
5758
#
5859
# This one is not just hygiene. Jackson's fixed-version ranges are per-line, and
5960
# an automated bump can move you INTO a vulnerability: CVE-2026-59889 was
6061
# introduced in 2.18.0 and fixed only in 2.18.9, and CVE-2026-54518 applies to
6162
# 2.21.0-2.21.3 only. The root pom greenlights CVE-2026-54518 as a false positive
62-
# *because this module resolves 2.18.9*; a bump into 2.21.0-2.21.3 would make that
63+
# *because this module resolves a 2.18.x*; a bump into 2.21.0-2.21.3 would make that
6364
# finding real while the exclusion silently hid it. Leaving the LINE is therefore a
6465
# hand-managed decision, and the exclusion list gets re-checked when it happens.
6566
#
6667
# Allow patches within the pinned line; block minor/major - the same shape as the
6768
# micrometer entry below, but here the patch allowance is load-bearing rather than
6869
# tidy. This started out blanket, which was wrong in a way worth recording: every
6970
# danger named above is a LINE change (2.18 -> 2.21), while a patch inside 2.18 is
70-
# the exact thing we are waiting for. The root pom's TEMPORARY exclusions for
71-
# CVE-2026-68497 and CVE-2026-19032 retire when jackson-databind 2.18.10 reaches
72-
# Central - a patch bump IS the retirement mechanism - and a blanket ignore
73-
# suppressed the only PR that would have announced it. Neither id is in OSV, NVD or
74-
# GHSA, so no Dependabot alert can arrive instead: the ignore was covering the one
71+
# the exact thing we are waiting for. The worked case: the root pom's TEMPORARY
72+
# exclusions for CVE-2026-68497 and CVE-2026-19032 retired when jackson-databind
73+
# 2.18.10 reached Central (astubbs/parallel-consumer#445, 2026-09-04) - a patch bump
74+
# WAS the retirement mechanism - and while this entry was blanket it suppressed the
75+
# only PR that would have announced it. Neither id is in OSV, NVD or GHSA, so no
76+
# Dependabot alert could arrive instead: the ignore was covering the one
7577
# notification we were relying on. bin/check-cve-exclusions.sh is the backstop for
76-
# when that happens anyway. See docs/inflight/deps-cve-backlog.md.
78+
# when that happens again. See docs/inflight/deps-cve-backlog.md.
7779
- dependency-name: "com.fasterxml.jackson:jackson-bom"
7880
update-types:
7981
- "version-update:semver-minor"

AGENTS.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,7 @@ is untracked (a whole triage doc was once written duplicating `docs/refactoring.
7575
| [`docs/building.md`](docs/building.md) | A build failed in a way that reads like a broken repository: the fresh-clone recipe, why the Truth assertion classes are generated rather than committed, and which invocations skip the generator |
7676
| [`docs/logging.md`](docs/logging.md) | Changing a logback file, adding a log stream, or wondering why a logging change had no effect - the two test profiles and how to prove which one loaded |
7777
| [`docs/inflight-tool.md`](docs/inflight-tool.md) | Querying the repo across every ref - worked examples for `bin/inflight.mjs`, and why each working-tree answer is wrong |
78+
| [`docs/grooming.md`](docs/grooming.md) | Running a vetting sweep of `docs/inflight/` - when, how it is split between agents, the dispatch prompt, and where PROPOSED markers and the release gating list are consolidated |
7879
| [`docs/testing.md`](docs/testing.md) | Writing or debugging tests: suite split, **why a run prints nothing and the flag that fixes it**, the ambient probe autopsy, the quarantine lane, the chaos suite, shared test utilities |
7980
| [`docs/ci.md`](docs/ci.md) | CI is red, or you are changing a workflow: what each workflow does, the self-hosted lanes, how to fetch a failed job's log |
8081
| [`docs/investigating.md`](docs/investigating.md) | Past the prior-art checks and into diagnosis: control arms, instrumentation traps, reporting rates |

bin/check-inflight-tags.sh

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ for f in docs/inflight/*.md; do
6666
# valid. Introduced exactly that way - merging a branch whose notes predated a state reword
6767
# appended the stale block underneath the corrected one rather than replacing it, and the gate
6868
# said "83 note(s) valid" over a note that was both `deferred - parked` and `parked - deferred`.
69-
for tag in type impact state labels; do
69+
for tag in type impact state labels vetted; do
7070
# No `|| echo 0`: grep -c PRINTS 0 and exits 1 on no match, so the fallback appends a
7171
# second line and the numeric test below errors on "0\n0".
7272
n=$(grep -c "<!-- inflight-$tag:" "$f" 2>/dev/null); n=${n:-0}
@@ -136,6 +136,27 @@ for f in docs/inflight/*.md; do
136136
if [ -n "$state" ] && ! grep -q ' - ' <<<"$state"; then
137137
note "$f \"$(note_title "$f")\": inflight-state '$state' has no reason. Use '<state> - <why>'"
138138
fi
139+
140+
# THE VETTED MARKER IS A DATE AND WHAT WAS CHECKED, or it is not one. `bin/inflight.mjs vet`
141+
# reads it with VETTED_RE in bin/lib/inflight-tags.mjs, which requires exactly `YYYY-MM-DD - <what>`;
142+
# a marker that regex cannot parse reads there as "never vetted" - silently undoing the vet it
143+
# records - so the shape is refused here, in front of whoever wrote it. The date is not checked
144+
# against a calendar: a typo in the day is a wrong record, which no script can tell from a
145+
# right one, but a marker with no date at all is a marker that says nothing.
146+
# THE MARKER FORM ONLY, `<!-- inflight-vetted:` - not the bare field name, which appears in prose
147+
# whenever a note quotes the grep that lists PROPOSED markers. The first cut matched the bare
148+
# name and failed the sweep's own working note for citing the command; the duplicate check above
149+
# already keys on the comment opener, and this follows it.
150+
vetted=$(sed -n 's/.*<!-- inflight-vetted:[[:space:]]*\([^>]*\)-->.*/\1/p' "$f" | head -1 | sed 's/[[:space:]]*$//')
151+
# A '>' INSIDE THE MARKER IS THE SAME DEFECT THE STATE CHECK ABOVE GUARDS: VETTED_RE in
152+
# bin/lib/inflight-tags.mjs is `[^>]*` too, so `fixed by X -> Y` parses as empty here and as
153+
# "never vetted" there. Five markers hit this on the first sweep and were reported as "not
154+
# YYYY-MM-DD", which sent the writer looking at the date. Name the actual cause.
155+
if grep -q '<!-- inflight-vetted:' "$f" && [ -z "$vetted" ] && grep -q '<!-- inflight-vetted:.*>.*-->' "$f"; then
156+
note "$f \"$(note_title "$f")\": inflight-vetted text contains '>' - VETTED_RE in bin/lib/inflight-tags.mjs cannot parse that marker and \`bin/inflight.mjs vet\` would list the note as never vetted. Reword without '>'"
157+
elif grep -q '<!-- inflight-vetted:' "$f" && ! grep -qE '^[0-9]{4}-[0-9]{2}-[0-9]{2} - .+' <<<"$vetted"; then
158+
note "$f \"$(note_title "$f")\": inflight-vetted '$vetted' is not 'YYYY-MM-DD - <what was checked>'. docs/inflight/AGENTS.md -> \"Vetting a note\" owns the marker"
159+
fi
139160
done
140161

141162
# THE DOC AND THE LIB MUST LIST THE SAME VALUES, and until now nothing checked it. The sets live in

bin/inflight.mjs

Lines changed: 77 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -55,16 +55,17 @@ import { fileURLToPath } from 'node:url'
5555

5656
import { perfReport, perfStart } from './lib/perf.mjs'
5757

58-
import { INVALIDATING_WARNINGS, baseline, freshnessWarnings, refTips } from './lib/git.mjs'
58+
import { INVALIDATING_WARNINGS, baseline, firstAddedDates, freshnessWarnings, refTips } from './lib/git.mjs'
5959
import { cacheClear, cacheStatus, knownCaches } from './lib/cache.mjs'
60-
import { corpusIndex, drift, findNotes, prsByBranch, stranded } from './lib/notes.mjs'
60+
import { corpusIndex, drift, findNotes, numbersByValue, prsByBranch, stranded } from './lib/notes.mjs'
6161
import { DOC_AREAS, NOTES_DIR } from './lib/repo.mjs'
6262
import { branchView, commitGraph, trackingGap } from './lib/branches.mjs'
6363
import { loadCandidates, refactorWindow } from './lib/refactor-window.mjs'
6464
import { RANKED_GROUPS, rank, registerBlob, runFailure } from './lib/rank.mjs'
65+
import { baselineNotes, baselineTree, symbolCandidates, symbolsPresent, vet } from './lib/vet.mjs'
6566
import {
6667
formatBranch, formatCache, formatCoverage, formatDrift, formatFind, formatFlakes, formatRefactorWindow, formatSlowest,
67-
formatRank, formatStranded, formatTimeline, formatWarnings,
68+
formatRank, formatStranded, formatTimeline, formatVet, formatWarnings,
6869
} from './lib/views.mjs'
6970
import {
7071
docsForBranch, docsSummary, docsUsage, forBranchSummary, forBranchUsage, headerSummary, headerUsage, indexDocs,
@@ -558,7 +559,8 @@ value is the reasoning attached to the order, which no computed scheme carries.
558559
if (!index.ok) return { ok: false, reason: `rank: ${index.reason}` }
559560
emit(formatWarnings(freshnessWarnings(index.baseline, index.refs.length)))
560561
const prs = prsByBranch()
561-
const r = rank(index, { prs, register: registerBlob(index), group })
562+
// ONE `git log` for every note's first-added date - the age the rows are ordered by.
563+
const r = rank(index, { prs, register: registerBlob(index), ages: firstAddedDates(`${NOTES_DIR}/`), group })
562564
if (!r.ok) return { ok: false, reason: `rank: ${r.reason}` }
563565
emit(formatRank(r))
564566
// EVERYTHING THAT DID RUN IS ALREADY EMITTED, and only then does the run report that it
@@ -570,6 +572,77 @@ value is the reasoning attached to the order, which no computed scheme carries.
570572
return { ok: true }
571573
},
572574
},
575+
{
576+
name: 'vet',
577+
summary: 'which open notes on the baseline to re-read first, and what about each looks stale before you open it',
578+
when: 'before a vetting sweep of docs/inflight, and before deciding what a release still gates on',
579+
usage: `Usage: bin/inflight.mjs vet every open note on the baseline, unvetted first
580+
bin/inflight.mjs vet --area <prefix> one area - the filename prefix, e.g. ci, test, core, bug
581+
bin/inflight.mjs vet --all deferred and closed notes too
582+
bin/inflight.mjs vet --ref <ref> read that ref instead of the baseline - a sweep branch before it lands
583+
584+
The worklist for a vetting sweep. Every open note on the baseline, partitioned by whether it carries
585+
an \`<!-- inflight-vetted: YYYY-MM-DD - what was checked -->\` marker, ordered by the index's group
586+
order and then OLDEST FIRST by the date the note was first added on any ref - and annotated with the
587+
cheap staleness signals: every fork number it cites is merged or closed, the number in its filename
588+
is settled, a cited path or symbol no longer resolves on the baseline, a delete-when line.
589+
590+
A SIGNAL IS A REASON TO OPEN THE NOTE, NEVER A VERDICT. A note cites a merged pull request because
591+
that is where the problem was found; a symbol is missing because the note proposes it. Nothing here
592+
closes a note. docs/inflight/AGENTS.md -> "Vetting a note" names the five outcomes and the marker;
593+
docs/grooming.md is the procedure for a whole sweep - follow it rather than re-deriving one.
594+
595+
Reads the baseline's blobs, never the working tree, and only the baseline: a note that exists only
596+
on a branch is vetted by that branch's merge. \`rank\` is the every-ref view.
597+
598+
bin/inflight.mjs vet
599+
bin/inflight.mjs vet --area bug`,
600+
run: (args, emit) => {
601+
const known = new Set(['--area', '--all', '--ref'])
602+
const unknown = args.filter((a) => a.startsWith('--') && !known.has(a))
603+
if (unknown.length) return { ok: false, reason: `vet: unknown option(s): ${unknown.join(', ')} - known: --area <prefix>, --all, --ref <ref>` }
604+
// THE SAME THREE GUARDS `rank` MAKES FOR `--impact`, for each valued flag: a missing
605+
// value, a repeated flag, and a positional that would otherwise be dropped silently.
606+
const valued = {}
607+
for (const flag of ['--area', '--ref']) {
608+
const at = args.indexOf(flag)
609+
if (at >= 0 && (args[at + 1] === undefined || args[at + 1].startsWith('--'))) {
610+
return { ok: false, reason: `vet: ${flag} needs a value after it` }
611+
}
612+
if (args.filter((a) => a === flag).length > 1) {
613+
return { ok: false, reason: `vet: ${flag} given more than once - which one did you mean?` }
614+
}
615+
valued[flag] = at >= 0 ? args[at + 1] : null
616+
}
617+
const consumed = new Set(Object.values(valued).filter((v) => v !== null))
618+
const stray = args.filter((a) => !known.has(a) && !consumed.has(a))
619+
if (stray.length) {
620+
return { ok: false, reason: `vet: takes no positional argument(s): ${stray.join(', ')} - did you mean --area ${stray[0]}?` }
621+
}
622+
const area = valued['--area'] !== null ? valued['--area'].replace(/-$/, '') : null
623+
const listed = baselineNotes({ ref: valued['--ref'] })
624+
if (!listed.ok) return { ok: false, reason: `vet: ${listed.reason}` }
625+
if (valued['--ref'] === null) emit(formatWarnings(freshnessWarnings(listed.baseline, 1, { invalidatingOnly: true })))
626+
const tree = baselineTree(listed.baseline)
627+
const symbols = symbolsPresent(listed.baseline, symbolCandidates(listed.notes))
628+
const v = vet(listed.notes, {
629+
numbers: numbersByValue(),
630+
tree,
631+
symbols,
632+
ages: firstAddedDates(`${NOTES_DIR}/`),
633+
area,
634+
all: args.includes('--all'),
635+
baseline: listed.baseline,
636+
})
637+
emit(formatVet(v))
638+
// THE SIGNALS THAT COULD NOT FIRE ARE A FAILED RUN, after everything that did run is
639+
// emitted - the rank shape. An unvetted list with no signals because gh was down reads
640+
// exactly like one where nothing is stale, and exit 2 is what tells them apart.
641+
if (!v.treeOk || !v.symbolsOk || !v.agesOk) return { ok: false, reason: 'vet: a git read failed - the answer above is incomplete' }
642+
if (!v.numbersOk) return { ok: false, reason: `vet: ${v.numbersReason} - the settled-number signals never ran` }
643+
return { ok: true }
644+
},
645+
},
573646
{
574647
name: 'refactor-window',
575648
summary: 'whether a file this repo means to decompose is cheap to decompose right now',

bin/lib/cache.mjs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,11 @@ const cacheDir = () => process.env.PC_INFLIGHT_CACHE_DIR
3838
const POLICY = {
3939
'prs.json': { maxAgeMs: 24 * 60 * 60 * 1000, cacheEmpty: true },
4040
'pr-branch.json': { maxAgeMs: 6 * 60 * 60 * 1000, cacheEmpty: false },
41+
// Every issue and pull-request NUMBER with its state, for `vet`'s "everything this note cites
42+
// is settled" signal. A day, like `prs.json`: a number that closes today reads as open until
43+
// tomorrow, which under-reports staleness rather than inventing it. Empty is not cached - a
44+
// repository with no numbers at all is a failed listing, not an answer.
45+
'numbers.json': { maxAgeMs: 24 * 60 * 60 * 1000, cacheEmpty: false },
4146
// Codecov's recorded test history. Ten minutes because a burst of `inflight codecov` queries in
4247
// one session should cost one fetch, while a CI run finishing mid-session still shows up. Empty
4348
// is not cached: an empty corpus would read as "no flakes recorded" - a false negative in the

bin/lib/git.mjs

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -510,3 +510,46 @@ export function freshnessWarnings(base, refCount, { invalidatingOnly = false } =
510510
}
511511
return warnings
512512
}
513+
514+
/**
515+
* The date each path under `pathspec` was FIRST ADDED, on any ref - `YYYY-MM-DD`, one process.
516+
*
517+
* AGE IS FIRST-ADDED, NEVER LAST-TOUCHED. `git log -1 -- path` answers "when was this last
518+
* edited", and on this repository that answer is the same for nearly every note: the package rename
519+
* of 2026-08-26/27 rewrote all of them, so last-touched separates nothing. First-added is the age a
520+
* reader means when asking which note has been waiting longest - and it is the one the rename sweep
521+
* cannot move.
522+
*
523+
* `--all`, because a note is born on the branch that produced it and reaches the baseline later, so
524+
* the baseline's own history dates the merge, not the note. `--diff-filter=A` keeps only the commits
525+
* that added the path; the earliest wins across every ref. A path renamed keeps the date of its
526+
* current name only - `--follow` is per-path and would be one process each, which this function
527+
* exists to avoid; the notes directory forbids renames for its own reasons, so the miss is small.
528+
*
529+
* `{ok, dates}` for the reason every batch here carries the flag: a failed log and a pathspec that
530+
* matches nothing both yield an empty map, and only one of them is an answer.
531+
*
532+
* @returns {{ok: boolean, dates: Map<string, string>}} path -> the earliest add date
533+
*/
534+
export function firstAddedDates(pathspec) {
535+
const specs = Array.isArray(pathspec) ? pathspec : [pathspec]
536+
// `-z` on the name list AND a NUL before the date, so a path containing a newline or a quote
537+
// cannot be mistaken for a boundary - the same reason treeEntries reads `-z`.
538+
const res = exec('git', ['log', '--all', '--diff-filter=A', '--name-only', '-z', '--format=%x00%cs', '--', ...specs])
539+
if (!res.ok) return { ok: false, dates: new Map() }
540+
const dates = new Map()
541+
// With `-z` git ends the commit header with NUL and the name list with NUL, so the stream is
542+
// `<NUL><date><NUL>\n<path><NUL>...` per commit: every NUL-separated token is either a date or a
543+
// path with git's own leading newline, and a path can never look like a date. Read it as one
544+
// token stream with the date carried forward, rather than guessing at record boundaries.
545+
let current = null
546+
for (const raw of res.out.split('\0')) {
547+
const token = raw.replace(/^\n/, '')
548+
if (!token) continue
549+
if (/^\d{4}-\d{2}-\d{2}$/.test(token)) { current = token; continue }
550+
if (current === null) continue
551+
const seen = dates.get(token)
552+
if (seen === undefined || current < seen) dates.set(token, current)
553+
}
554+
return { ok: true, dates }
555+
}

0 commit comments

Comments
 (0)