[TF24 hydraulics] Score the scenario gateway on two axes, not a match rate (#572)

[dfalster]

Closes #572. Implements options 3 + 1 — your stated preference: keep status as the numerical-viability gate, add persistence as the primary ecological metric, retire the bare match rate as the headline. No CSV rewrite, no re-blessing.

The problem, restated from the measurements

classify_scm_run() calls a run a success on finite && total > 0. With birth_rate = 1 (what build_scenario() sets) offspring_production is R0, so that is far weaker than it looks: five of the eight hydraulic scenarios return R0 between 2e-15 and 6e-14 — numerically extinct — and every one was recorded as "persisted". All eight "succeeded", none of the five expected failures failed, and only one replaced itself.

What changed

scenario_summary() now reports two labelled axes, plus agreement demoted:

Axis	Columns	Question
Numerical viability	n_ran, n_crashed, viability_rate	Did the model run? (what the CSV's "Model failure" means, #549/#550)
Ecological persistence	n_persists, persistence_rate	Does the strategy replace itself, R0 ≥ 1? (via persists_at(), #570)
CSV agreement	n_match, match_rate, expected-fail/success breakdown	Does the outcome match the CSV's expectation?

Reflected in scripts/run_scenario_gateway.R and inst/reports/scenario_scorecard.Rmd. On the #570-head numbers the gateway now prints:

  Numerical viability : 8 / 8 ran (100%), 0 crashed
  Persistence (R0>=1) : 1 / 8 persist (12%)
  CSV agreement       : 3 / 8 (38%) — agreement with the crash-era
                        expectations, not a quality score

Viability is counted from the recorded crashed flag rather than from match, so an expected failure that crashes counts against viability while still counting for agreement — the two axes have to be able to disagree or there is nothing to separate.

Why agreement is demoted rather than dropped

With the crashes fixed (#546/#552/#554) the match rate mostly measures how well the CSV's crash predictions have aged. The tell is that fixing the model lowered it, 5/8 → 3/8. It is still worth reporting; it is not a quality score.

Not done, deliberately

Option 2 (rewriting the Expectation column to be about persistence). That discards the crash-prediction record and forces a re-blessing. notes/plan-tf24-scenario-framework.md records the decision, that option 2 was declined, and that the old expectations are recoverable from git history if it is ever taken.

status and outcome are untouched, so the blessed-baseline diff is still defined on the same columns and the baseline is not re-blessed. Scorecards predating the persists / crashed columns (including the blessed baseline) still summarise rather than error.

Threshold caveat, recorded

R0 ≥ 1 is evaluated at a fixed birth rate, so it carries the same density-dependence artefact that inflated TF24t-vs-TF24 R0 gaps in unrelated work. All scenarios use birth_rate = 1, so the comparison is internally consistent; persists_at(total, finite, threshold = 1) takes the threshold as an argument for when that stops being true.

Done-when

• Scorecard reports viability and persistence as separate, clearly-labelled axes
• Headline in run_scenario_gateway.R and the report is no longer a bare match rate
• Establish and quantify expected hydraulic behaviour for TF24/TF24F (scenario gateway) #557's stale 5/8 corrected (its banner now records the two-axis reading), and notes/plan-tf24-scenario-framework.md updated with the option taken
• Baseline not re-blessed — the change is deliberately confined to reporting, and the reason is recorded in the note

Tests

test-scenario-eval.R: 50 expectations, green. Two new blocks — the axes are reported separately and can disagree; viability is counted from crashed, not from the match.

🤖 Generated with Claude Code