Routing: add deterministic dispatch complexity scoring

[Necmttn]

Problem

ax already has deterministic subagent routing through route-dispatch, routing-table.json, ax dispatches --candidates, and ax routing tune. The current live decision is mostly agent type + regex over the Agent description.

That misses an important signal: the actual Agent prompt's structural complexity. A tiny bounded implementation and a long multi-file implementation can share the same description prefix, but should not necessarily receive the same model-routing advice.

Wayfinder Router has a useful pattern to borrow: an offline, deterministic prompt-complexity scorer that extracts structural features, reports explanations, and can be calibrated against local traffic without a model call.

Reference: https://github.com/itsthelore/wayfinder-router

Proposal

Add an ax-native deterministic dispatch-complexity scorer, then backtest it before letting it affect live hook behavior.

Phase 1: pure scorer, no behavior change

Add packages/hooks-sdk/src/prompt-complexity.ts with an Effect-free, synchronous API safe for the hook fire path:

• strip leading YAML frontmatter
• count structural prompt features:
  • word count
  • Markdown heading count + max heading depth
  • list item count
  • Markdown link count
  • fenced code block count
  • table row count
• compute lexical features but weight them 0 by default:
  • reasoning terms
  • math symbols
  • constraint terms
  • question count
• normalize with saturating feature caps
• return score: 0..1, raw features, and top feature contributions

Keep lexical features reported-but-off by default. Wayfinder's own blind eval says lexical hard-word cues do not generalize well.

Phase 2: CLI explain/backtest surface

Add a read-only command, e.g.:

ax routing score --text "..." --json
ax routing score --from-call <tool_call_id> --json

Extend dispatch analytics/backtest code to parse the Agent prompt from tool_call.input_json in addition to the existing model extraction.

Use historical dispatch rows to report:

• complexity score distribution by child model
• score distribution by routing class
• expensive inherited dispatches with low score
• regex route-down matches with high score
• estimated savings if low-score unmatched dispatches had routed down
• false-positive risk examples where high score or judgment terms should keep strong model

Phase 3: calibration

Add a calibration command only after the scorer has useful backtest signal:

ax routing calibrate --days=90 --objective=knee

Use ax graph data as labels/proxies:

• child model and cost from session_token_usage
• dispatch metadata from spawned + Agent tool_call.input_json
• advice follow-through from the advice ledger
• repair/churn/failure proxies where available

Output proposed thresholds and examples. Do not auto-apply thresholds initially.

Phase 4: optional hook integration

Only after backtest/calibration, let complexity influence resolveDispatchModel:

1. judgment guard still wins: review/design/audit stays strong
2. explicit model handling remains unchanged
3. agent-type rules remain high precedence
4. regex routing classes remain the operator intent layer
5. complexity can suppress/soften route-down advice when the actual prompt is structurally heavy
6. low-complexity unmatched dispatches become tune candidates, not automatic live route-downs

Non-goals

• Do not add Wayfinder as a Python runtime dependency.
• Do not add an OpenAI-compatible model gateway to ax.
• Do not replace routing-table.json regex/agent-type routing.
• Do not turn lexical hard-word cues on by default.
• Do not make live hook behavior depend on this until backtested.

Why this fits ax

This deepens the existing routing module instead of adding a second routing system:

• @ax/hooks-sdk already owns the hook-safe routing seam.
• route-dispatch already makes advisory, quota-aware model-routing decisions.
• ax routing tune already mines local dispatch history.
• ax has richer local labels than a generic router: cost, model choice, advice outcome, repair episodes, and user corrections.

The first PR should be zero-risk: pure scorer + tests + CLI/backtest visibility, no hook behavior change.

Acceptance criteria

• Pure scorer module exists in packages/hooks-sdk with no DB, no Effect runtime requirement, and no network/model call.
• Unit tests cover frontmatter stripping, fenced-code exclusion, saturation, contribution math, and lexical-features-off-by-default behavior.
• ax routing score explains a prompt's score and top contributors.
• Dispatch analysis can score historical Agent prompts from tool_call.input_json.prompt.
• Backtest output identifies candidate thresholds/examples without changing live hook advice.

---

Generated with ax.