llm proxy: route on LlmBeginEx.model, extracted at llm server via bounded JsonPath (design)

[jfallows]

Summary

Design note for llm proxy (a later slice, not part of v1's M1–M6 milestones): llm server extracts model from the request body and carries it on LlmBeginEx alongside the already-shipped dialect field, so llm proxy routes purely on stream metadata — dialect + model — rather than re-inspecting request content at the routing layer.

Why this shape

Same pattern mcp(proxy) already uses: its routable key (tool name) also lives in the JSON-RPC body, not headers/path, yet binding-mcp decodes it during normal protocol handling and exposes it as a matchable route attribute — the proxy layer itself never does content-expression routing. Following that precedent means llm proxy does not need a generic ${content.*} route-expression namespace as a prerequisite (the design handoff doc originally flagged that as an engine prerequisite for the proxy slice) — route when: blocks match on LlmBeginEx fields, the same mechanism every other proxy-shaped binding already uses.

Constraint this has to respect

Unlike MCP, llm has a stricter invariant from M2's round-trip identity test: same-dialect passthrough must decode framing and re-encode with no payload parsed, at all. So model extraction can't become "just parse the body like MCP does" — every request would silently lose that guarantee, not only proxy-routed ones. Extraction has to stay a narrow, bounded pull of just the model field — common-json's JsonPath (bounded, resumable, single-field extraction) is the right tool, not a general parse.

Scope (when this slice is picked up — not now)

• LlmBeginEx gains a model field, populated by llm server via bounded JsonPath extraction, without buffering or fully parsing the request body.
• llm proxy route conditions match on both dialect and model from LlmBeginEx, mirroring mcp(proxy)'s toolkit/tool route-matching pattern.
• Routing is intra-dialect (multiple backends/models within one dialect family) — cross-dialect model-name equivalence mapping (e.g. treating a gpt-4o request as routable to a Claude backend) is out of scope; this matches the existing note that failover/load-balancing is constrained to same-dialect backends.

Explicitly not doing now

Do not add model to LlmBeginEx in the M1 idl issue (#2476). It's deliberately deferred there ("needed for proxy routing, out of scope for this slice") — adding it speculatively ahead of llm proxy actually existing would be exactly the kind of speculative field the codebase's own conventions warn against. This issue exists to record the design decision for when llm proxy is picked up, not to trigger the idl change today.

Depends on / relation

• Builds on LlmBeginEx's existing dialect-only shape (binding-llm.spec: llm.idl — LlmBeginEx, LlmDataEx, LlmFlushEx union #2476) and llm server dialect detection (llm server: dialect detection (path + header signals, fixed-dialect mode) #2483).
• Not part of v1 (M1–M6); scoped to the deferred llm proxy slice referenced in the original design handoff doc's "Later slices" section (model-based routing, quota enforcement, failover).