refactor(cache): cached-inference layer out of analyze_audio.py (T051, parts 1-2) - #538
Merged
Conversation
satra
changed the title
satra
marked this pull request as ready for review
July 27, 2026 12:58
satra
added a commit
that referenced
this pull request
Jul 27, 2026
Every job in tests.yaml is gated on `pull_request.draft == false`, but `ready_for_review` was missing from the trigger types. So a PR opened as a draft and later marked ready never re-triggered the workflow: its cpu-tests, pre-commit and tutorial-cpu-tests stayed "skipping" indefinitely, while build-and-deploy (a separate workflow with no draft gate) showed green. That combination is actively misleading — a reviewer sees one passing check next to skipped tests and reasonably reads it as "tests are fine". Found on #538, where ten commits sat unvalidated: `gh pr ready` flipped the draft flag but ran nothing. Adding the event is enough; the existing draft gate then evaluates false and the jobs run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
satra
changed the base branch from
20260722-175022-scene-quality-utterance
to
alpha
July 27, 2026 23:15
…dio.py (T051, part 1) First slice of T051. Moves the content-addressable cache layer from scripts/analyze_audio.py into src/senselab/utils/tasks/cached_inference.py: CACHE_SCHEMA_VERSION, serialize, canonical_params, cache_key, align_cache_key, transcript_signature, cache_lookup, cache_store, senselab_version, sync_cache_with_schema_version. The script drops 180 lines (2492 → 2312) and imports them under their historical names, so no call site changed. Key derivation is deliberately byte-identical, so the existing artifacts/analyze_audio_cache/ entries stay valid. That is pinned rather than asserted: the golden digests captured from the pre-refactor script are now test constants, and the script was re-checked after the move to confirm it still produces them. A future drift shows up as "the cache was invalidated" instead of a silent hours-long re-run. The layer had no unit tests before; it now has 26, covering key stability and sensitivity per keyed field, ASR-vs-alignment key independence, store/lookup round-trip, corrupt-entry-as-miss, tensor serialization, and all four branches of the schema-version sync (fresh / match / mismatch-wipe / unreadable marker). wrapper_version_hash intentionally stays in the script and still hashes the script's own source, which keeps current cache keys intact. Its re-scoping is coupled to moving the _stage_* functions to workflows/audio_analysis/stages.py — that is the remaining half of T051 and carries the documented invalidation-semantics change, so it belongs in its own commit rather than being smuggled in here. Verified: 26 new tests pass; analyze_audio_test.py 76 passed / 1 skipped; ruff check + format clean (287 files); mypy clean on all three files. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…m (T051, part 2) Moves run_task / run_task_cached / run_alignment_cached out of scripts/analyze_audio.py into utils/tasks/cached_inference.py, where the cache primitives they drive already live. Script: 2312 → 2243 lines. run_task_cached and run_alignment_cached were literal duplicates — the original docstring even said "identical control flow to run_task_cached; the distinction is semantic". They now share one `run_cached` implementation, with the two public names kept as thin wrappers so call sites still read as "task" vs "alignment" and the log line still distinguishes them. The semantic difference that actually matters is preserved and unchanged: alignment keys come from align_cache_key and its provenance carries transcript_sha / language / parent_asr_cache_key, keeping the alignment cache independent of the parent ASR cache. Behavior is unchanged, including the non-obvious contract that a *failed* task is never stored — so a fixed backend or a senselab upgrade retries instead of replaying a cached error. That was previously implicit in two places; it now has explicit tests for both entry points. 8 new tests (34 total in the module): ok/failed capture, miss-runs-and-stores, hit-skips-execution, cache-disabled-never-stores, failures-not-cached for both runners. Verified: 84 passed / 1 skipped across analyze_audio_test.py + the cache module; ruff and mypy clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…on, schema 2 (T051, part 3)
Acts on an architectural review of the remaining T051 work, now that backward
compatibility with artifacts/analyze_audio_cache/ is explicitly not required.
Three substantive changes rather than a mechanical move.
**1. audio_signature was left behind, and it matters most.** Parts 1-2 moved the
cache primitives but not the function that produces their join key. It is the
contract between summary.json (`passes[label].audio_signature`) and each entry's
`provenance.audio_signature`: adaptive/loop.py reads the former,
adaptive/interventions.py::build_cache_index indexes on the latter, and a
mismatch makes cache-replay escalation silently never fire. Both now come from
one function. Typed structurally (Protocol on waveform/sampling_rate) because
utils/ must not import audio/ — that would invert the dependency direction.
**2. wrapper_hash → code_version.** Hashing the CLI script's own source was the
wrong tool: editing a comment, a docstring, or letting ruff-format rotate a line
invalidated every cached model result. Worse after the stages move, since one
stages.py would share a hash across six unrelated stages. The parameter (and the
hashed payload key) is now a caller-supplied behavior identifier. The script
still passes its source hash for the moment; STAGE_VERSIONS replaces it when the
stages land, which is where the semantics actually change.
**3. CACHE_SCHEMA_VERSION 1 → 2.** The key payload changed shape, so every
existing entry is unreadable by construction. Bumping is strictly better than
`rm -rf`: sync_cache_with_schema_version wipes on marker mismatch, so *every*
host invalidates automatically rather than just this machine. Verified — the
local cache reported "schema_version 1 → 2; wiped 152 stale entries".
Also moved `write_json` (same operation as cache_store, shares serialize).
Tests: the golden cache_key digest is deleted, not updated. It existed solely to
assert "no invalidation", a goal now explicitly void; keeping a pinned digest
would ossify the payload shape for no reason. Replaced with a determinism check.
The sha256("hello world") transcript pin stays — that is a real external
constant. Two script tests that used the old kwarg were moved here rather than
shimmed, carrying their FR-010 / FR-024 citations so traceability survives.
40 tests in the module.
Verified: 88 passed / 1 skipped; ruff + mypy + codespell clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…steps 2-3)
Prep for the stages move: three helpers in analyze_audio.py were library-grade
code in the wrong place. Script 2220 → 2089 lines. No behavior change beyond one
deliberate consolidation (below).
**extract_temporal_features + _flatten_feature_dict → tasks/features_extraction/temporal.py.**
This is a model-touching capability (multi-backend windowed extraction over
extract_segments + extract_features_from_audios), not workflow plumbing, so it
belongs beside api.py / opensmile.py / torchaudio_squim.py — the same reasoning
that moved fusion to speech_to_text_ensemble (T050) and WER to
speech_to_text_evaluation (T049).
**pick_dispatch_model → model_for_task in utils/data_structures/model.py.**
Pure model-id → provider-class routing, so it belongs in the module defining
those subclasses, next to check_hf_repo_exists. It also has three consumers (two
stages plus the enhancement path), which rules out a workflow-private home. Noted
in the docstring: the diarization branch duplicates diarize_audios's internal
dispatch — pre-existing, worth collapsing eventually.
**_safe → safe_model_id, consolidating two divergent copies.** There were two:
char-wise in the script ("a--b" → "a__b") and collapse-and-strip in
labelstudio.py ("a--b" → "a_b"). They produce identical output for every model id
in the current defaults, since none contains adjacent non-alphanumerics — so
unifying now is behavior-neutral, whereas leaving it would eventually surface as
a filename-vs-LS-track-name divergence. Adopted the collapsing semantics (also
never returns empty, which LS from_name requires). labelstudio.py's copy turned
out to be dead code with no call sites.
Tests: the two orphaned script tests moved rather than being kept alive with
re-export shims — that would preserve exactly the coupling this removes.
safe_model_id gains a table over the real shipped model ids (proving the merge is
behavior-neutral) plus an explicit case documenting the collapse rule where the
old variants disagreed. model_for_task gains routing + unknown-task coverage.
Verified: 13 passed in model_test; 177 passed / 5 skipped across the script,
utils and labelstudio suites; ruff + mypy clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…x dict (T051, step 4) Introduces the two frozen types the stages will take, replacing `(args: argparse.Namespace, ctx: dict[str, Any])`. Nothing consumes them yet — landed separately so the contract is reviewable before the 400-line move. **StageContext** — the run environment (pass label, audio signature, device, cache/output dirs) plus the cache-key and provenance derivation that previously lived in untyped closures on the ctx dict (`ctx["key"]`, `ctx["prov"]`). Those closures were the real problem, not the dict: closed over five variables, untypeable, unmockable, invisible to a reader. `out_dir=None` gives a headless mode — `write_sidecar` becomes a no-op — which is what the adaptive loop wants when it runs a stage without a run directory. **PassPlan** — what to run, with absence meaning skip (empty tuples, `None` model ids) rather than a CLI-shaped `skip` set. `args.skip` mixes pass-level and post-pass concerns and is *mutated after parsing* on the no-speech triage path, so translating it into an explicit plan is the script's job. **STAGE_VERSIONS replaces source hashing.** Per-stage hand-bumped integers, keyed by the task strings already in the cache key, surfaced as `"asr@1"` so a cache entry's provenance is readable. `stage_code_version` raises on an undeclared stage rather than defaulting to 1, so a new stage can't silently inherit another's invalidation fate. Pinned by a test, so a bump is a visible two-line diff. Two things the tests caught that are worth noting: - **The import-weight guard found a real leak.** `cached_inference` imported torch at module top for `serialize`'s isinstance check, so importing the *config types* pulled in torch and transformers. torch is now imported lazily inside `serialize` (a sys.modules lookup after the first call), and a subprocess test asserts `stage_context` loads with neither in `sys.modules`. `DeviceType` is behind TYPE_CHECKING for the same reason — `device_label` only reads `.value`. - **`device_label` maps None → "auto", not "cpu".** It goes into the cache key and the provenance, so collapsing "let senselab choose" into a concrete device would change every key and make provenance claim something the run never specified. Highest-value test: `test_provenance_joins_to_build_cache_index` round-trips a provenance block through `cache_store` and asserts the entry resolves in `adaptive/interventions.py::build_cache_index`. That join (on audio_signature) is how cache-replay escalation finds prior results; if it ever breaks it does so silently — no error, no log. 18 tests here, 58 with the cache module. ruff + mypy clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… (T051, step 5) The core of T051. `scripts/analyze_audio.py` 2089 → 1667 lines; the pipeline is now importable instead of CLI-only, which is the precondition for T040. New `workflows/audio_analysis/stages.py`: six `stage_*` functions plus `run_pass` and four private helpers. Each stage takes `(audio, ctx: StageContext, *, knobs)` and **returns** its summary fragment rather than mutating a shared dict. The fragment keys are unchanged, because they are a published contract — presence.py, compute.py, identity.py, utterance.py, global_summary.py and the adaptive interventions all read `pass_summary["asr"]["by_model"]`, `["ast"]`, `["yamnet"]`, `["features"]["result"]`, `["ppgs"]`. `run_pass` moved too, which T051 didn't ask for but should have: the ctx construction *was* the cache/provenance layer, so leaving it behind would have kept library-grade code in the CLI and left T040 with no importable entry point. **stage_alignment now takes `asr_by_model` explicitly.** It used to reach into `ctx["summary"]["asr"]["by_model"]` — an invisible must-run-after-stage_asr dependency and a latent KeyError. As a parameter it's type-checked, and a caller can align a *cached* ASR block it never produced, which is exactly the adaptive loop's escalation path. There's a test for that case. **wrapper_version_hash is deleted.** Cache keys now carry `STAGE_VERSIONS` per-stage tokens (`"asr@1"`). summary.json and the parquet/disagreements provenance report `stage_versions` instead of `wrapper_version_hash`. Net effect: editing CLI plumbing no longer invalidates the model cache; the counterpart obligation is bumping a stage's version when its output shape changes. The script keeps two ~30-line adapters, `_pass_plan(args)` and `_stage_context(...)`, so `argparse` stops at the CLI boundary. `_pass_plan` is called *after* triage because `main` mutates `args.skip`/`args.ppg` on the no-speech path — building it earlier would run diarization and ASR on silence. Two more transitive helpers than the task listed: `_classification_windows` and `_classification_window_top1` are called by `_scene_agreement`/`_top1` and live in harvesters.py. Ruff's F821 caught them, which is the argument for moving code in one lint-verified step rather than by hand. 22 new stage tests, all model-free (each monkeypatches the tasks/ call), pinning the failures that would otherwise be silent: - stage_features returns the LIVE row dict while its JSON sidecar carries the "see features/*.parquet" placeholder — return the sidecar shape instead and every loudness/quality column becomes None rather than raising; - stage_ppg's key is "ppgs" (consumers accept both spellings, so a rename degrades to null signals); - an empty PassPlan runs no expensive stage, so a flipped default can't burn the model suite on a silent clip; - cache hit does not re-invoke the model. Verified: 381 passed / 8 skipped across workflows + script + utils; mypy clean on all 286 files (the moved code is newly type-checked — scripts/ is excluded from mypy and ruff, so this is its first pass); ruff, format and codespell clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ers (T051) The end-to-end run caught a real bug the unit tests could not: `_pass_plan` read `args.align_asr`, but the flag is `--no-align-asr` (store_true), so the attribute is `args.no_align_asr`. Every invocation died with AttributeError before running a single stage. Worth noting *why* the suite missed it. The library tests exercise `PassPlan` directly, and the script tests never touched the adapters — so the one place argparse attribute names are transcribed by hand had no coverage at all. That is exactly the seam a refactor like this introduces. Five adapter tests added, covering what the library tests structurally cannot: every attribute `_pass_plan` reads actually exists on a parsed Namespace; `--no-align-asr` is inverted correctly; `--skip` translates to empty tuples and None ids; a post-triage mutated Namespace yields a plan that runs neither diarization nor ASR (the no-speech path); and `_stage_context` records the resolved source path, "auto" device label, and a 64-char signature. Verified end-to-end on tutorial_audio_files/audio_48khz_mono_16bits.wav: diarization + AST + YAMNet + features + ASR all ran, sidecars written, summary.json reports `stage_versions` (no `wrapper_version_hash`), and the ASR provenance carries `code_version: asr@1`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`run_adaptive_loop` could only ingest a *finished* run directory — it read
summary.json and the nine uncertainty parquets off disk. T040 needs it callable
mid-run, from the PassHarvest objects analyze_audio just produced, with no
parquet round-trip. `VoteStore.from_harvests` already existed and documents
itself as "the in-process integration point for analyze_audio"; nothing called it
from the loop.
Adds `harvests=` and `summary=` parameters. Both ingest paths now share the loop:
artifact-driven (unchanged default, used by scripts/adaptive_loop.py) and
in-process.
**The parity check is reported as "skipped", not as passing.** This is the part
worth reviewing: `VoteStore.parity_check` re-aggregates every bucket and compares
against the value *stored in the parquet*. On the in-process path those values
don't exist yet, so every bucket would come back "compared: 0, mismatches: 0" —
which reads as a clean pass while proving nothing about the harvest/aggregate
split. It now emits `{"status": "skipped", "reason": ...}` so a reader can't
mistake absence of evidence for evidence. The artifact path keeps the real check.
Also filters harvests to the passes the summary reports as completed, so a failed
enhancement pass can't contribute votes for a stream with no duration_s.
3 tests: in-process ingest emits round-1 artifacts; parity reports skipped with
its reason; a failed pass present in harvests but absent from the summary is
ignored.
This is the foundation, not all of T040 — the nine CLI flags
(--max-rounds/--policy/--budget-*/--max-region-rounds/--region-top-n/
--reserve-asr-models/--enable-overlap-separation/--no-adaptive-outputs), the
summary.json `adaptive` block, and the `--skip comparisons` warning are still to
come, as is golden-compat verification that `--max-rounds 1` reproduces today's
outputs byte-for-byte.
Verified: 208 passed / 3 skipped across the workflow suites; ruff + mypy clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ummary block (T040) Completes T040. analyze_audio now runs the adaptive loop within the same process as the pipeline, on the harvests the uncertainty parquets were just built from. CLI (contracts/cli.md): --max-rounds, --policy, --budget-medium, --budget-heavy, --max-region-rounds, --region-top-n, --reserve-asr-models, --enable-overlap-separation, --no-adaptive-outputs. Plus the promised warning that --skip comparisons disables the belief store and therefore all rounds >= 2. Policy precedence is packaged default < --policy file < CLI flags, and `policy_hash` is recomputed *after* merging. That last part matters for provenance: two runs with the same --policy but different --budget-heavy must not claim the same hash. Unset flags resolve to None and are dropped, so a flag left alone overrides nothing rather than resetting to a CLI default. Harvests reach the loop through a new `harvests_out=` parameter on `compute_uncertainty_axes` — an out-parameter rather than a fourth return value, so no existing caller's tuple arity changes. **Golden-compat verified**, not assumed. `--max-rounds 1` against `--no-adaptive-outputs` on the same clip: uncertainty parquets byte-identical (3/3 for a single-pass run), labelstudio_tasks.json and labelstudio_config.xml identical, summary.json with no keys added or removed and no pre-existing value changed, and rounds/ + final/ present only in the adaptive run. disagreements.json differed in exactly one field: `generated_at`. Two deviations from the contract, both documented in tasks.md: - `--enable-overlap-separation` is described there as a "v2 U4 rule (off by default)". No U4 rule exists; the shipped overlap rule is `I4_overlap_detection`, which the packaged policy already enables. The flag therefore forces I4 on, which only has an effect against a policy file that turned it off. Wiring it to a nonexistent rule, or flipping the shipped default to off, would both have been worse. - The loop's return had no `policy_hash`; the summary block wanted one. Added it to the return (it was already computed) rather than re-reading final/iterations.json. I initially wrote the block against invented keys (`rounds_run`) and caught it by checking the actual return dict — it now reads rounds / run_state / n_interventions_fired / n_words_fused / parity_check.status. A real end-to-end run (whisper-tiny, --max-rounds 2 --budget-heavy 0) produced 2 rounds, 2 interventions fired, 19 words fused, and a full final/ including the RTTM sidecar. 9 new tests: override precedence, None-dropping, hash sensitivity, empty-override identity, and the five flag→policy mappings. Verified: 398 passed / 15 skipped; mypy clean (286 files); ruff, format and codespell clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every job in tests.yaml is gated on `pull_request.draft == false`, but `ready_for_review` was missing from the trigger types. So a PR opened as a draft and later marked ready never re-triggered the workflow: its cpu-tests, pre-commit and tutorial-cpu-tests stayed "skipping" indefinitely, while build-and-deploy (a separate workflow with no draft gate) showed green. That combination is actively misleading — a reviewer sees one passing check next to skipped tests and reasonably reads it as "tests are fine". Found on #538, where ten commits sat unvalidated: `gh pr ready` flipped the draft flag but ran nothing. Adding the event is enough; the existing draft gate then evaluates false and the jobs run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…T051b) Follows the architecture review's T051b: the Label Studio export was library-grade code sitting in the CLI, while the uncertainty-track builders it sits beside already lived in labelstudio.py. Ten functions (~289 lines) move, so the whole LS surface is now in one module. Script 1730 → 1491 lines; only the two public entry points (build_labelstudio_task / build_labelstudio_config) are imported back. **Found a second divergent duplicate — this one had a real consequence.** `_asr_has_timestamps` existed in *both* the script and harvesters.py, with different semantics: - harvesters (strict): needs a non-null `start` on a chunk or the line. - script (loose): `start is not None or len(chunks) > 0`. They disagree on a chunked-but-untimed transcript: the loose version calls it timestamped. `stage_alignment` used the loose one to decide whether to skip alignment — so exactly the input alignment exists to fix would be skipped and never get timestamps. Meanwhile `resolve_asr_result`, two functions away in harvesters.py, has always used the strict version, so the codebase disagreed with itself about the same transcript. Consolidated on the strict semantics, with the reasoning in the docstring and a test pinning the disagreement case. Behavior change: a chunked-but-untimed transcript is now aligned instead of silently skipped. That's the intended direction, but it is a change. `seg_attr` was also duplicated — byte-identical this time, so dropping the copy is a pure cleanup. Two script tests that reached for the moved privates were relocated with the code rather than kept alive by re-export shims. Verified: 403 passed / 15 skipped; mypy clean (286 files); ruff + format clean. End-to-end re-run against the pre-move baseline: labelstudio_tasks.json and labelstudio_config.xml byte-identical, uncertainty parquets byte-identical (3/3). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ntics doc.md is the pdoc-rendered page for this package, and it described none of the T051/T040 surface — no mention of stages.py, stage_context.py, cached_inference, STAGE_VERSIONS, or the adaptive CLI. Anyone arriving at the module would still believe the pipeline is CLI-only. Adds a section covering the importable entry point (run_pass / StageContext / PassPlan), where the cache layer now lives, and the two things about it that are non-obvious enough to be worth stating outright: - STAGE_VERSIONS is hand-bumped per stage, and the obligation is explicit — bump when a stage's stored outcome shape changes. It replaced a source hash that rotated on comment edits and reformats. - the adaptive loop's in-process path reports parity as "skipped", not passing, because there are no stored parquet values to compare against yet. Also records the flag precedence (default < --policy < CLI) and that `--max-rounds 1` is the verified golden-compat mode. The Python example was executed, not just written: the three imports resolve through the package's lazy exports. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds the fine-grid presence re-analysis rule, registered ahead of I4 so the planner can satisfy I4's declared "else fires P2 first" dependency. Reuses backends.overlap_posteriors — it already runs segmentation-3.0 on the crop and returns the `speech` track alongside `overlap`, so P2 needs no new backend. Replacement presence votes enter at `scope=region:<id>`, superseding the coarse round-1 voters without deleting them (the store keeps both, later scope wins, decision log stays auditable), and `overlap_posterior` is written on covered rows, which is what lets I4 subsequently run "light (reuses P2 output)". **Two bugs the live run caught that the unit tests did not.** Worth recording because the second is a testing lesson, not just a fix: 1. The trigger read vote payloads off the belief row. Rows only carry `contributing_sources` (names) — payloads live in the VoteStore. So `coarse` was never visible, `n_active_votes` was 0, and the rule could never fire. Now reads `store.active_votes(...)`. 2. `frame_instability` never reached `row_meta`, so the second trigger branch was dead. Plumbed through `VoteStore.from_harvests`, where the harvest bucket carries it. The artifact path still lacks it — presence.parquet has no such column. The unit tests passed through both because the fixture fabricated rows with a `model_votes` key that does not exist on a real row. The fixture now builds a real `VoteStore`, so that class of drift can't hide again. **P2 has NOT been observed firing end-to-end**, and the cause is a threshold boundary rather than a defect: - presence `aggregated_uncertainty` is a decisiveness measure (`1 − |2p−1|`) that peaks at 0.554 even on the noise degradation fixture, so no presence region is seeded at the default `theta_high: 0.66` — verified directly: 0 regions at 0.66, 1 region at 0.30. - with the threshold lowered the region appears and the trigger evaluates, but `coarse_share` comes out at **0.4967** — a hair under the contract's `≥ 0.5`. Both numbers are recorded in tasks.md. Retuning `theta_high` for the presence axis or the coarse-share threshold changes behavior for every axis or every clip respectively, so I left that as a human decision rather than tuning until the rule fired. 10 tests (registration order, axis/cost, both trigger branches, non-presence rejection, inactive-vote handling, region-scoped vote replacement, overlap emission, posterior failure). 229 passed / 3 skipped; mypy clean (286 files); ruff + codespell clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…043)
Ran both env-gated end-to-end harnesses against real full-pipeline runs for the
first time, on macOS ARM64. Results recorded in prototype-results.md.
**T039 — SC-001: 5/5 (100%)**, threshold 70%. All five degradation variants
analyzed; injected span [1.969, 4.921] overlapped by 4 proposed regions in every
variant.
The interesting part is *how* it passes: only `silence` was **improved** (an
intervention moved a bucket by > 0.05). The other four satisfy the criterion via
its "or explained" branch — the loop marks the span irreducible instead of
reducing its uncertainty. Legitimate as written, but it means SC-001 currently
evidences that the loop *recognizes* degradation, not that it *repairs* it. Flagged
for a decision on whether the criterion should separate those.
Region proposal required `thresholds.theta_high: 0.30` via `--policy`: at the
packaged 0.66 no presence region is seeded on any variant, because presence
uncertainty is a decisiveness measure (`1 − |2p−1|`) peaking at 0.554. Same root
cause as the T041 note.
**T038 — SC-005: pass.** Golden (`--max-rounds 1 --no-adaptive-outputs`) vs
candidate (`--max-rounds 3`), both two-pass so all 9 parquets exist: every parquet
value-equal at atol=1e-12, ASR + diarization result payloads identical.
**The T038 harness was itself broken and could never have passed.** It enumerated
`rglob("uncertainty/*.parquet")`, which matches `<pass>/uncertainty/<axis>.parquet`
(6) but cannot match the cross-pass deltas at
`uncertainty/raw_vs_enhanced/<axis>.parquet` (3) — so its own `assert checked >= 9`
was unreachable for any input. Now filters `rglob("*.parquet")` on paths containing
an `uncertainty` component.
That is the second task in this spec marked complete that had never been run — the
first being T038/T039 themselves being written without execution. Worth noting the
pattern: an env-gated test that nobody has supplied the env for is indistinguishable
from a passing one.
T044 also marked complete: `VoteStore.from_harvests` has both its unit test and its
first in-process caller (T040) as of this branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Scope is "planner + regions first" per the task: regions.py, policy.py, convergence.py, loop.py. **Implemented as TypedDict, not dataclasses — a deliberate deviation from the task wording.** `Region` is written to `rounds/<n>/regions.json` and read back by plot.py, ls_final.py and the T039 harness; `PlannedIntervention` lands in final/iterations.json. A dataclass would need to_dict/from_dict at each of those boundaries while the dict remained the actual wire format — a second representation to keep in sync rather than a replacement. `plan_round` also adds `status`/`error`/`intervention_id` *after* constructing the record, which a frozen dataclass cannot express and a mutable one gains nothing from. The defect class here is key typos and wrong value types, not mutation, and TypedDict catches exactly that across every existing consumer with zero runtime or serialization change. **mypy found three latent defects the dict soup was hiding**, which is the return on the exercise: - `exec_status` is written by loop.py after a rule runs but was never declared anywhere — invisible to every reader of the record. - `region_id` was read as if always present while being assigned only after ranking in `propose_regions`; now seeded at construction so the key is honestly required. - a test fixture constructed `Region`s without `n_buckets`. `AxisName` is now a shared literal so `AXES` keeps its narrowing through `propose_regions` instead of widening to `str` at the first call site. `LoopContext` and `InterventionRule` are left untyped on purpose. `ctx` is a genuinely heterogeneous bag mutated across rounds (store, state, policy, cache index, per-round scratch); typing it usefully means restructuring it, which is a different change from annotating it. 229 passed / 3 skipped; mypy clean across 287 files; ruff + codespell clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t configurable Closes the open question from T041/T043 with a decision rather than leaving it dangling in two task notes. Presence `aggregated_uncertainty` peaks at 0.554 against a `theta_high` of 0.66, so no presence region is seeded by default and P2 is unreachable. Not being tuned around now: the available ground truth is a single annotated clip, and fitting a seeding threshold to one example fits noise. The threshold belongs to a benchmark set, and that same exercise should decide whether presence needs its own value at all — its aggregator returns a decisiveness measure (`1 − |2p−1|`) that does not share a scale with the distance-based identity/utterance aggregators. The only standing requirement is that it be adjustable without a code change, and it already is: `thresholds.theta_high` deep-merges from a `--policy` file over the packaged default. Verified that a one-key file applies, leaves `theta_low` at its default, and changes `policy_hash` so two runs differing only in this threshold stay distinguishable in provenance. The recorded T039 runs used exactly this at 0.30. Deliberately NOT adding per-axis threshold machinery yet. That is the likely shape once a benchmark exists, but building tuning knobs for a deferred decision is how config surfaces rot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…otnotes (T045) All four items T045 lists were already recorded in the contract's "Implementation notes" section (2026-07-24). The problem was that the *per-rule sections still contradicted that section*: `## U2_reserve_escalation` said "Cost: heavy" and listed the "reserve family not already majority" guard as though it existed, when the code has `cost: medium` and `guard: None`. A reader landing on the U2 section got the wrong answer unless they happened to scroll to a footnote 50 lines later. Corrected inline at the source, with the drafted-but-unimplemented guard marked as such rather than deleted, so the intent stays legible. Two discrepancies the earlier pass missed, now documented on their own sections: - **P2_fine_posteriors** is implemented (T041) but⚠️ unreachable at the default policy: seeding needs `aggregated_uncertainty ≥ theta_high` (0.66) while presence uncertainty is a decisiveness measure (`1 − |2p−1|`) that peaked at 0.554 across 242 buckets on the noise fixture. Also recorded what it actually does — reuses `backends.overlap_posteriors`, so segmentation-3.0's native ~16.9 ms hop *is* the fine grid and there is no separate `fine_hop_s: 0.01` knob as drafted. - **U4_overlap_separation** has⚠️ no implementation at all. The `--enable-overlap-separation` flag added in T040 maps to the shipped `I4_overlap_detection`, not to separation — worth stating on the U4 section itself, since the flag name reads like it enables this rule. Leaves T042 as the spec's only open item. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ns (T042) Closes the adaptive spec's last open item. The RTTM sidecar, `member_labels`/ `overlap` and `transcript.json.language` landed earlier; the three `presence.parquet` columns from contracts/final-outputs.md were still absent — found by reading a real full-pipeline run's output rather than assuming. - `presence_confidence` — the calibrated P(speech). Kept *beside* the existing `p_voice` rather than renaming that column, so anything already reading this file keeps working. - `elected_stream` — S1's per-region stream choice, falling back to the fusion stream where no per-region election ran. - `overlap_posterior` — written by I4/P2 when per-class segmentation posteriors were available, `None` elsewhere. The column is emitted either way so the schema stays stable for readers. Verified on a full default run: 242/242 non-null for the first two, 97/242 for overlap_posterior (only the spans I4 actually touched). Also a test asserting all three exist even when values are None — the failure mode this guards is a column silently disappearing, which a value-based test would miss. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ence replaces p_voice
Two gaps, both reported from a real run rather than found by tests.
**The summary figure was never produced by the in-process path.**
`build_adaptive_timeline` was only ever called from `scripts/adaptive_loop.py`, so
T040's integration emitted every adaptive artifact except the one a human actually
looks at — `final/timeline.png` is documented in CLAUDE.md and was simply absent.
It now runs at the end of `run_adaptive_loop`, so both ingest paths get it; the
path is returned in the loop's result, recorded in `summary.json → adaptive.timeline`,
and printed by analyze_audio. Best-effort: a plotting failure warns and does not
fail the run. The standalone script keeps its own call, which additionally applies
the ground-truth overlay only it supports.
**presence_confidence replaces p_voice outright** in `final/presence.parquet`,
rather than sitting beside it as I first wrote it. Nothing on the way to alpha needs
backwards compatibility, and two names for one quantity is how schemas rot.
That rename needed care about *which* parquet each consumer reads, and I got it
wrong once: `evaluate.py` reads `final/presence.parquet` (so it moves to the new
name), but `plot.py` reads the per-round `rounds/<n>/belief/presence.parquet`, which
legitimately still has `p_voice`. Repointing both broke the figure with
`KeyError('presence_confidence')` — caught by running it, then reverted for plot.py
only. The two files are different schemas that happen to describe the same axis.
Verified on a full default run: timeline.png emitted (5 rows: ground-truth
placeholder, presence with uncertainty band, identity round-1-vs-final, utterance
with intervention annotations, per-word confidence), and 18 interventions recorded.
230 passed / 3 skipped; mypy clean (287 files); ruff + format clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…daptive timeline **Cache pruning.** `senselab_version` and `code_version` are both *inside* the cache key, so a senselab release orphans every entry and a `STAGE_VERSIONS` bump orphans that stage's — but only `CACHE_SCHEMA_VERSION` ever triggered a wipe. The directory therefore grew monotonically across releases, and a cache that looked healthy could be entirely dead weight. It happens to be 100% live right now only because the 1→2 schema bump wiped its predecessors; that was luck, not design. `prune_unreachable_entries` now removes entries whose recorded provenance can no longer be keyed, and `sync_cache_with_schema_version` calls it on every run — so "the cache only holds reachable entries" becomes an invariant rather than a coincidence. Entries *without* provenance are kept: absence of evidence isn't evidence of staleness, and a hit on them is still correct. **Spectrogram row** on `final/timeline.png`, placed first so the acoustic evidence sits above the beliefs derived from it — a reviewer can now see *why* a span is uncertain (noise, silence, overlap) rather than only that it is. On the audio_48khz sample the broadband haze across the whole spectrum immediately explains the elevated presence uncertainty. Rendered with numpy's rfft rather than librosa so the plot adds no dependency, 70 dB dynamic range, and wrapped so any failure leaves an annotated empty axis instead of losing the figure. 5 pruning tests (version drift, stage bump, retired task, no-provenance, live-cache no-op). 394 passed / 8 skipped; mypy clean (287 files); ruff + codespell clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
satra
force-pushed
the
20260726-t051-cached-inference-stages
branch
from
July 27, 2026 23:19
6ffcc48 to
321e155
Compare
wilke0818
added a commit
that referenced
this pull request
Jul 28, 2026
#536 (scene-quality/utterance) + #538 (cached-inference refactor) merged to alpha, so their model-load sites now live here too. Route them through the helpers (cherry-picked from the hf-cache-on-536 integration branch), guard-driven: In-process (SHA-pin / snapshot-path): - voice_activity_detection/frame_posteriors.py (use the SHA ensure_hf_model returns) - workflows/audio_analysis/adaptive/audio_io.py (SepFormer via snapshot path) - workflows/audio_analysis/adaptive/backends.py (ASR pipeline via load_hf_resilient) Subprocess (hf_subprocess_env): - scene_quality/brouhaha.py (gated; token forwarded, online fallback preserved) - speech_to_text/crisperwhisper.py (online fallback preserved) Guard-test allowlist updated to cover all five; PR #1 now covers every HF load site on alpha. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adaptive Phase-8 T051, split out per the implementation audit's own "(Own PR)" note. Precondition for T040.
Based on
20260722-175022-scene-quality-utterance(PR #536) rather thanalpha, since T051 touchesscripts/analyze_audio.py, which #536 also modifies. Retarget toalphaonce #536 lands.Done — the cache layer is fully extracted
scripts/analyze_audio.py: 2492 → 2243 lines (−249).Part 1 (
e2216363) — cache primitives →src/senselab/utils/tasks/cached_inference.py:CACHE_SCHEMA_VERSION,serialize,canonical_params,cache_key,align_cache_key,transcript_signature,cache_lookup,cache_store,senselab_version,sync_cache_with_schema_version.Cache keys are byte-identical, so existing
artifacts/analyze_audio_cache/entries stay valid. Pinned by golden digests captured from the script before the move, then re-verified against the script after — a drift reads as "the cache was invalidated" rather than a silent hours-long re-run.Part 2 (
55e5e833) — the runners:run_task,run_task_cached,run_alignment_cached.run_task_cachedandrun_alignment_cachedwere literal duplicates (the original docstring said so). They now share onerun_cached, with both public names kept as thin wrappers so call sites still read as task vs alignment. The semantic difference that matters is untouched: alignment keys come fromalign_cache_key, and its provenance carriestranscript_sha/language/parent_asr_cache_key.Also made explicit: a failed task is never cached, so a fixed backend or senselab upgrade retries instead of replaying an error. That was implicit in two copies; now tested at both entry points.
The layer had zero unit tests; it now has 34.
Remaining — the stages move, and a design question
Not done here, deliberately. The remainder is
_stage_{diarization,scene,features,asr,alignment,ppg}→workflows/audio_analysis/stages.py, plus re-scopingwrapper_version_hashfrom "hash this script" to "hash the stage modules".Two reasons it's a separate unit rather than rushed into this PR:
It's 442 lines across 13 functions, not 6. The stages call seven script-level helpers —
pick_dispatch_model,extract_temporal_features,_scene_agreement,_asr_has_timestamps,_extract_transcript_text,_safe,write_json— and a library module cannot import from a script, so all of them must move too.It needs a signature decision. The stages are
(audio, args: argparse.Namespace, ctx: dict). Relocating that as-is is the lowest-risk mechanical move, but it puts anargparsedependency insidesenselab.audio.workflows— a CLI shape leaking into the library. The alternative is explicit parameters per stage, which is the better boundary but a larger change than "move these functions". Worth an explicit call before writing it.That part also carries T051's documented invalidation-semantics change: re-scoping the wrapper hash narrows invalidation to the code that actually shapes a stage's output, but rewrites every cache key once. Keeping it separate is what makes the zero-invalidation move above reviewable on its own — which is why
wrapper_version_hashstill lives in the script and still hashes its own source.Verification
analyze_audio_test.py: 84 passed / 1 skipped (combined run)cache_key/transcript_signaturedigests🤖 Generated with Claude Code
Version
Published prerelease version:
1.3.1-alpha.36Changelog
🐛 Bug Fix
Authors: 3