feat(router): add load-aware generate paths

[bvolpato]

Purpose

Extensions can expose generate-compatible endpoints beyond /inference/v1/generate. Unknown paths currently use transparent proxying, which bypasses typed request routing and cache-aware worker selection.

Add extra_generate_paths and --extra-generate-paths. Each configured path:

• validates with existing InferenceGenerateRequest schema
• extracts token_ids for cache-aware routing
• forwards request to same backend path
• uses typed retries, circuit breaking, and in-flight load accounting
• supports regular and prefill/decode routing

This lets integrations register endpoints without adding integration-specific routes or concepts to vLLM Router.

Compatibility

Configured paths retain transparent-proxy wire behavior:

• request JSON is forwarded semantically unchanged, including omitted optional fields, nested values, and unknown extension fields
• client-only headers remain isolated while configured backend authorization is preserved
• backend response status, headers, and body bytes are preserved
• configured requests use existing authorization and concurrency limiting
• built-in routes keep priority and cannot be overridden by extension configuration
• unconfigured paths retain existing transparent-proxy or disabled-proxy 404 behavior

Routing metadata and original JSON remain separate, so typed selection never rewrites payload shape. Prefill/decode routing receives explicit generate request kind instead of inferring it from payload fields.

Configured paths are validated before startup to reject duplicates and dynamic patterns. Extra paths dispatch from unmatched-route fallback, so built-in routes remain defined by Axum route table without duplicate reserved-path inventory.

Cancellation-safe global load accounting is intentionally separate in #200. Related consumer dependency/startup update: NovaSky-AI/SkyRL#1988.

Test Plan

• Register /custom/v1/generate against two delayed workers.
• Start one shared-prefix request, then send another while first remains active.
• Verify cache-aware routing sends one request to each worker.
• Compare complete forwarded JSON object, including omitted stream, nested values, priority, and unknown fields.
• Verify backend path, authorization isolation, response status, headers, and body.
• Verify configured paths use concurrency limiting and cannot override built-ins.
• Verify disabled transparent proxy still returns 404 before auth for unknown paths.
• Verify prefill/decode request-kind separation and nested sampling-limit mutation.
• Validate Python and Rust configuration surfaces.

Test Result

• cargo fmt --all -- --check: passed.
• cargo clippy --all-targets --all-features -- -D warnings: passed.
• cargo test --all-features: passed, including 487 library tests and 51 API endpoint tests.
• .venv/bin/pytest py_test/unit -q: 109 passed, 4 skipped.
• uvx black --check py_src py_test: passed.
• uvx ruff check py_src py_test: passed.
• git diff --check: passed.

---

Essential Elements of an Effective PR Description Checklist

• Purpose documented.
• Test plan documented.
• Test results documented.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2c35714f5e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8d88d153c8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".