Merge pull request #11 from bgyss/codex/dkcr-hd-scaffold

Document and automate DKCR HD validation workflow

Author: bgyss

.github/pull_request_template.md

@@ -0,0 +1,14 @@
+## Summary
+-
+
+## Files Touched
+-
+
+## Testing (required)
+- `cargo test`
+- Result:
+
+If tests are skipped, include explicit approval and the reason.
+
+## Open Questions
+-

AGENTS.md

@@ -30,7 +30,9 @@ SwitchRecomp is a preservation-focused static recompilation research project. Th
 ## Testing Guidelines
 - Use `cargo test` for Rust workspace tests.
 - Test files live under `crates/*/tests/` or inline in modules.
-- Always run the full suite after changes unless the user explicitly says not to.
+- Always run the full suite after changes.
+- Record the full-suite command and outcome in the PR Testing section and in status updates.
+- Do not skip tests unless the user explicitly approves; record the approval and reason in the PR.
 
 ## Back Pressure Hooks
 Pre-commit hooks provide fast feedback; `prek` is a drop-in replacement that reads the same `.pre-commit-config.yaml`.
@@ -47,6 +49,8 @@ Pre-commit hooks provide fast feedback; `prek` is a drop-in replacement that rea
 No established commit conventions are present yet. Until standards are set:
 - Use clear, imperative commit messages (e.g., "Add SPEC-070 OS services draft").
 - PRs should include a short summary, a list of files touched, and any open questions.
+- PRs must include a Testing section with the full suite (`cargo test`) and results.
+- Do not use "Testing: Not run" unless the user explicitly approved skipping tests.
 - Link related issues if available.
 
 ## Research & Sources

PLANS.md

@@ -23,10 +23,6 @@ This file tracks implementation work derived from specs that do not yet have a c
 - SPEC-180 XCI Title Intake
 - SPEC-190 Video-Based Validation
 - SPEC-200 DKCR HD First-Level Milestone (macOS/aarch64)
-- SPEC-210 Automated Recompilation Loop
-- SPEC-220 Input Replay and Interaction Scripts
-- SPEC-230 Reference Media Normalization
-- SPEC-240 Validation Orchestration and Triage
 
 ## SPEC-000: Project Charter and Ethics
 Outcome
@@ -299,20 +295,13 @@ Exit criteria (from SPEC-180)
 Outcome
 - Validate the recompiled output against a reference gameplay video without emulator traces.
 
-Note
-- DKCR validation is paused until the automation loop, input replay, and normalization specs land (SPEC-210/220/230/240).
-
 Work items
 - [x] Define a reference timeline for the first level and store it in `reference_video.toml`.
 - [x] Implement a capture workflow for macOS/aarch64 runtime output.
 - [x] Add a comparison step that computes video and audio similarity metrics.
 - [x] Generate a `validation-report.json` with pass/fail and drift summaries.
 - [x] Document manual review steps for mismatches.
 
-External prerequisites (see `docs/dkcr-validation-prereqs.md`)
-- Absolute paths to reference and capture artifacts (video or hashes).
-- Confirmed first-level start and end timecodes.
-
 Exit criteria (from SPEC-190)
 - A single run produces a validation report for the first level.
 - Similarity metrics are stable across two consecutive runs.
@@ -322,80 +311,97 @@ Exit criteria (from SPEC-190)
 Outcome
 - Produce a macOS/aarch64 static recompilation of DKCR HD that reaches and plays the first level.
 
-Note
-- DKCR validation is paused until SPEC-210/220/230/240 are implemented.
-
 Work items
 - [x] Complete XCI intake for the DKCR HD title (SPEC-180 inputs and outputs).
 - [x] Identify required OS services and implement or stub them in the runtime.
 - [x] Implement the minimal GPU translation path needed for the first level.
 - [x] Create a per-title config and patch set for DKCR HD.
 - [x] Run video-based validation against the first level (SPEC-190).
 
-External prerequisites (see `docs/dkcr-validation-prereqs.md`)
-- Absolute paths to DKCR reference and capture artifacts.
-- Confirmed first-level start and end timecodes.
-
 Exit criteria (from SPEC-200)
 - The macOS/aarch64 build boots and reaches the first playable level.
 - First-level gameplay matches the reference video within defined tolerances.
 - No proprietary assets or keys are stored in the repo or build outputs.
 
-## SPEC-210: Automated Recompilation Loop
-Outcome
-- Provide a one-command automation loop for intake, build, capture, and validation.
-
-Work items
-- [x] Define `automation.toml` schema and validator.
-- [x] Implement an orchestrator CLI that runs intake -> lift -> build -> run -> capture -> validate.
-- [x] Emit a deterministic `run-manifest.json` with step timings and artifact hashes.
-- [x] Add resume/caching logic keyed by input hashes.
-- [x] Add integration tests using non-proprietary fixtures.
-
-Exit criteria (from SPEC-210)
-- One command runs the full loop and produces a run manifest and validation report.
-- Re-running with identical inputs yields identical artifacts.
-- Proprietary assets remain external.
-
-## SPEC-220: Input Replay and Interaction Scripts
-Outcome
-- Deterministic input playback aligned to reference timelines.
-
-Work items
-- [x] Define `input_script.toml` schema with events and markers.
-- [x] Implement input script loader and runtime playback module.
-- [x] Add tools/tests for deterministic playback and alignment.
-- [x] Document authoring and replay workflows.
-
-Exit criteria (from SPEC-220)
-- Input scripts replay deterministically across two runs.
-- Playback order is stable for simultaneous events.
-- Markers align to reference timecodes.
-
-## SPEC-230: Reference Media Normalization
-Outcome
-- Normalize reference video/audio into a canonical, comparable format.
-
-Work items
-- [x] Define canonical reference profile (resolution, fps, audio).
-- [x] Implement normalization workflow and metadata capture.
-- [x] Update `reference_video.toml` schema to record normalization details.
-- [x] Add hash generation tests for normalized outputs.
-
-Exit criteria (from SPEC-230)
-- Reference media can be normalized deterministically.
-- Hashes for normalized outputs are stable across runs.
-
-## SPEC-240: Validation Orchestration and Triage
-Outcome
-- Automated validation with structured reports and triage summaries.
-
-Work items
-- [x] Define `validation-config.toml` and report schema extensions.
-- [x] Implement triage summary generation (drift, likely causes).
-- [x] Integrate validation orchestration into the automation loop.
-- [x] Add tests for report determinism and failure summaries.
-
-Exit criteria (from SPEC-240)
-- Validation runs emit deterministic reports and triage summaries.
-- Failures include actionable context and artifact references.
+## Future Work Notes: DKCR HD Video Validation (SPEC-190/200)
+Requirements to finalize:
+- Define the capture spec (resolution, fps, constant frame rate, audio rate/bit depth/channels, container/codec).
+- Define alignment parameters (search window, pre-roll, min match window, segment length).
+- Document drift thresholds for video metrics and audio correlation with pass/warn/fail tiers.
+- Capture runtime configuration (performance mode, GPU backend, resolution scale, frame pacing, input map).
+- Record hashes for XCI intake outputs, module.json, manifest.json, and bundle manifest.
+- Record tool versions (recomp-cli, ffmpeg, capture tooling) per validation run.
+- Track each run with a validation artifact index and store reports under a dated output root.
+- Record host metadata (hardware model, OS version, GPU driver/toolchain versions).
+
+Required per-run metadata (external):
+- Run id, start/end timestamps, duration, and title/config identifiers.
+- Git commit hash, build manifest hash, bundle manifest hash, and intake output hash.
+- Runtime config snapshot and capture config snapshot with absolute paths.
+- Alignment parameters and similarity thresholds used for the run.
+
+Validation artifact index fields (external):
+- Paths and sha256 hashes for reference capture, recompiled capture, and alignment outputs.
+- Paths and sha256 hashes for reports, summary json, and run logs.
+- Pointers to the run metadata record and per-stage timing summary.
+
+Artifact paths and naming (external only):
+- Reference captures: `/Volumes/External/validation/dkcr-hd/reference/YYYY-MM-DD/`.
+- Recompiled captures: `/Volumes/External/validation/dkcr-hd/recompiled/YYYY-MM-DD/`.
+- Alignment outputs: `/Volumes/External/validation/dkcr-hd/alignment/YYYY-MM-DD/`.
+- Reports: `/Volumes/External/validation/dkcr-hd/reports/YYYY-MM-DD/validation-report.json`.
+- Run metadata: `/Volumes/External/validation/dkcr-hd/runs/YYYY-MM-DD/run.json`.
+- Artifact index: `/Volumes/External/validation/dkcr-hd/index/YYYY-MM-DD/run-index.json`.
+- Provenance entries should include the external paths and file hashes.
+
+Timeline checklist (per validation run):
+- Day 0: capture reference footage and compute hashes.
+- Day 1: capture recompiled footage under matching settings.
+- Day 1-2: run alignment + metric generation, record drift windows.
+- Day 2: manual review of flagged windows and finalize report.
+- Day 3: summarize deltas and file follow-up issues.
+
+## Future Work Notes: Automation Ideas (Local + Cloud)
+Local automation ideas:
+- A `scripts/run_recomp_pipeline.sh` wrapper that runs intake, lift, pipeline, build, and validation
+  with consistent logging and a summary JSON.
+- Cache build artifacts under `out/` with per-run timestamps and a manifest index.
+- Emit a single `run.log` plus `summary.json` so automation can surface failures quickly.
+- Add a `run.json` manifest with input hashes, config paths, tool versions, and stage timings.
+- Support `--resume` to reuse intake/lift outputs and `--clean` to purge temp outputs.
+- Create per-run working directories named `runs/YYYYMMDD-HHMMSS-shortsha/`.
+
+Cloud automation ideas:
+- Containerize the pipeline (Rust toolchain + hactool/hactoolnet + ffmpeg).
+- Use object storage for XCI intake outputs, validation artifacts, and reports.
+- Inject key material via a secrets manager; never write keys to persistent storage.
+- Run jobs on ephemeral workers with a per-run working directory and explicit cleanup.
+- Record container image digests and worker hardware profiles in run metadata.
+- Add artifact retention rules and a nightly prune job for stale runs.
+
+Agent-driven pipeline flow (candidate):
+1. Intake: validate provenance, run XCI extraction, emit module/manifest.
+2. Lift: decode and emit lifted module and segments.
+3. Build: run `recomp-cli run`, build emitted project, package bundle.
+4. Validate: capture and align video, run `recomp-validation`, emit report.
+5. Report: upload artifacts, summarize pass/fail and drift windows.
+6. Archive: write run metadata and validation artifact index entries.
+
+Dependencies to document:
+- Rust toolchain, `cargo`, `nix` (optional dev shell).
+- `hactool` or `hactoolnet` for XCI extraction.
+- `ffmpeg` for capture and alignment steps.
+- Storage and logging backends for automation outputs.
+- Capture tooling and codec requirements for the validation pipeline.
+
+## Future Work Notes: Automation/Docs Backlog
+Automation requests:
+- XciExtractor helper that wraps tool discovery, key validation, Program NCA selection, and deterministic output layout.
+- Validation artifacts index generator that emits a single JSON with paths, hashes, and run metadata pointers.
+- Playback comparison tooling that produces side-by-side video with timecode overlays and drift markers.
+- Run registry that aggregates per-run summary.json files into a searchable index.
+
+Documentation requests:
+- XciExtractor usage and key-handling guidance with expected output layout.
+- Validation artifact index schema and example run metadata record.
+- Playback comparison workflow with expected inputs and review checklist.

README.md

@@ -32,8 +32,12 @@ Legal and provenance policy:
 
 ## Samples and Flow Docs
 - `samples/memory-image/` shows the memory image initialization flow (segment blob + lifted module).
-- `docs/static-recompilation-flow.md` outlines a hypothetical macOS static recompilation flow and verification pipeline.
-- `docs/validation-matrix-template.md`, `docs/title-run-sheet-template.md`, `docs/thresholds/default.json`, `docs/batch-manifest-schema.md`, `docs/batch-manifest-schema.json`, and `docs/batch-pipeline-layout.md` are reusable validation templates.
+- `samples/validation/` contains non-proprietary validation artifact index templates.
+- `docs/static-recompilation-flow.md` outlines a hypothetical macOS static recompilation flow and verification pipeline (see the Real XCI intake section for external-tool usage).
+- `docs/xci-intake.md` documents the XCI intake workflow and mock extractor format.
+- `docs/validation-artifacts.md` defines validation artifact indexing, workflows, and dependencies.
+- `docs/validation-video.md` describes the hash-based video validation workflow.
+- `scripts/capture-validation.sh`, `scripts/capture-video-macos.sh`, `scripts/capture_video.sh`, and `scripts/validate_artifacts.sh` provide capture and validation helpers.
 
 ## Back Pressure Hooks
 These hooks add fast, consistent feedback to keep the repo autonomous and reduce review churn. Hooks are defined in `.pre-commit-config.yaml` and can be run with `prek` (preferred) or `pre-commit`.

crates/recomp-cli/src/automation.rs

@@ -1,3 +1,6 @@
+#![allow(dead_code)]
+// Automation scaffolding is intentionally compiled but not yet wired into the CLI.
+
 use recomp_pipeline::homebrew::{
     intake_homebrew, lift_homebrew, IntakeOptions, LiftMode, LiftOptions,
 };