Skip to content

HTML_VERIFICATION.md

Latest commit

 

History

History
120 lines (87 loc) · 4.73 KB

HTML_VERIFICATION.md

File metadata and controls

120 lines (87 loc) · 4.73 KB
Raw

HTML Verification Commands

Use these commands when validating the HTML rollout end to end.

For downstream rollout, dual-run, and rollback behavior around routed-family consumption, see docs/HTML_CONSUMER_CUTOVER.md.

Full local verification

Run the full repository verification surface, including tests and clippy:

bash scripts/html_verify.sh

This is the non-interactive command intended for CI or local pre-merge checks.

Selector-addressable regions and page-breaks

Verify that selector-anchored HTML regions emit only bounds/indices and that raw HTML page-break boundaries remain addressable without adding a synthesized page model:

bash scripts/selector_region_e2e.sh

The wrapper writes artifacts under artifacts/html-e2e/region/selector/, uses the existing HTML e2e harness, and records region_table_count, region_page_spans, region_as_of_dates, and region_byte_offsets_present in run.summary.json.

SOI region parity

Compare selector-bounded Schedule of Investments regions against legacy SOI metrics and emit mismatch details under artifacts/html-e2e/parity/<label>/.

Prepare a JSONL file with one row per filing. The committed CI fixture uses period_end, table_count, page_span, and holding_row_count:

{"path":"/abs/path/to/filing.html","period_end":"2025-09-30","table_count":1,"page_span":[5,5],"holding_row_count":1}

Then run:

bash scripts/soi_parity.sh \
  --definitions-dir rules \
  --legacy-results /tmp/soi-legacy.jsonl \
  --artifact-root artifacts/html-e2e \
  --label soi-parity

For an external real-filing corpus, pass --manifest /path/to/corpus.jsonl and --legacy-command-template 'python3 /path/to/extract_schedule_sections.py {path}'. The command must emit a JSON object with the same metric keys.

Legacy parity audit

Compare fingerprint family routing to a legacy route source and emit mismatch artifacts under artifacts/html-e2e/parity/<label>/.

With precomputed legacy routes

Prepare a JSONL file with one row per document:

{"path":"/abs/path/to/filing.html","legacy_family":"ares"}

Then run:

bash scripts/html_parity_audit.sh \
  --definitions-dir rules \
  --legacy-results /tmp/legacy-routes.jsonl \
  --artifact-root artifacts/html-e2e \
  --label committed-fixtures

Against an external legacy router

If the full 40+ filing corpus lives outside the repository, point the harness at an external manifest and invoke the existing Python router with a command template. {path} is replaced with the absolute document path for each file.

bash scripts/html_parity_audit.sh \
  --definitions-dir rules \
  --manifest /data/bdc/html_corpus.jsonl \
  --inventory tests/fixtures/html/inventory.json \
  --legacy-command-template 'python /path/to/fingerprint_schedule_family.py {path}' \
  --artifact-root artifacts/html-e2e \
  --label external-corpus \
  --diagnose-mismatches

Key artifacts

  • matrix/<label>/ — raw fingerprint matrix run produced by the shared HTML e2e harness
  • parity/<label>/parity.summary.json — overall parity counts and artifact pointers
  • parity/<label>/parity.mismatches.jsonl — file-level mismatches with observed family, legacy family, child-routing status, and diagnose artifact paths
  • parity/<label>/legacy.routes.jsonl — normalized legacy routing records used for the comparison

Reading progress and diagnose artifacts

  • stderr.events.json — parsed --progress stream from fingerprint; inspect this when a run appears to stall or when warning counts increase unexpectedly.
  • diagnostics.json — normalized fingerprint.diagnostics payloads for each stdout record, including attempted fingerprints, first failed assertions, near misses, and short-circuit context.
  • fixture.summary.jsonl — one row per document with route_resolved, child_routing_status, selected_child_fingerprint_id, matched_child_fingerprint_ids, refusal codes, and skip status.
  • family.summary.json — aggregated routed-family counts keyed by expected family and selected fingerprint ID.
  • run.summary.json — top-level counters such as ambiguous_route_count, selected_child_count, progress_event_count, warning_event_count, and the artifact file manifest for the run.

For mismatch triage, start with parity.summary.json, open the corresponding rows in parity.mismatches.jsonl, then follow any diagnose_artifact_dir pointer into the shared harness artifacts. The quickest signal is usually:

  1. fixture.summary.jsonl to see whether the route failed, skipped, or became ambiguous.
  2. diagnostics.json to identify which child assertion lost the route.
  3. stderr.events.json to confirm whether parser warnings or progress anomalies happened during the run.