Title: [Bug] DP intra-node GET proxies fail when worker URLs contain @rank (userinfo misinterpretation)
Description
When running with intra-node data parallel size > 1, worker registry URLs carry an @rank suffix (e.g. http://127.0.0.1:18100@0). The GET proxy path in proxy_get_request (src/routers/http/router.rs) forwards these URLs without stripping @rank:
let url = format!("{}/{}", worker_url, endpoint); // http://127.0.0.1:18100@0/v1/models
Per RFC 3986, @ in the authority part is parsed as userinfo, so reqwest sends this request to host 0 instead of 127.0.0.1:18100. The request fails and the router returns 500.
Affected endpoints (all via proxy_get_request)
GET /v1/models (breaks Swarm / AgentBench preflight)
GET /health_generate
GET /get_server_info
GET /get_model_info
Unaffected: POST /v1/chat/completions (already strips @rank via dp_utils::parse_worker_url and sends X-data-parallel-rank) and GET /health (send_health_check already strips via extract_dp_rank). DP=1 is unaffected since there is no @rank suffix.
Reproduction (uses the repo's own mock worker; verified with the release binary)
# 1. Start the repo's mock worker (py_test/fixtures/mock_worker.py)
python py_test/fixtures/mock_worker.py --host 127.0.0.1 --port 23340 > worker.log 2>&1 &
# 2. Start the router with DP=2 (creates workers http://127.0.0.1:23340@0 and @1)
RUST_LOG=vllm_router_rs=debug target/release/vllm-router \
--host 127.0.0.1 --port 8001 \
--worker-urls http://127.0.0.1:23340 \
--policy round_robin \
--intra-node-data-parallel-size 2 \
--health-check-interval-secs 1 \
--prometheus-port 7293 \
> router.log 2>&1 &
# 3. Wait for "Router ready", then hit the preflight endpoint:
curl -sS -i http://127.0.0.1:8001/v1/models
Observed output:
# router.log
Router ready | workers: ["http://127.0.0.1:23340@0", "http://127.0.0.1:23340@1"]
# GET /v1/models through router (DP=2)
HTTP/1.1 500
Request failed: error sending request for url (http://0.0.0.0/v1/models)
# contrast 1 — direct GET to worker: 200
curl http://127.0.0.1:23340/v1/models
{"data":[{"id":"mock","object":"model"}]}
# contrast 2 — POST through router (DP=2): 200
curl -X POST http://127.0.0.1:8001/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"mock","messages":[{"role":"user","content":"hi"}]}'
{"id":"cmpl-...","choices":[{"text":"ok",...}],"worker_id":"worker-23340",...}
# contrast 3 — GET through router with DP=1: 200
The 500 body shows the outbound URL became http://0.0.0.0/v1/models — the router sent the request to host 0 (port 80) instead of 127.0.0.1:23340; the worker never receives it. POST and DP=1 paths succeed, isolating the bug to the @rank suffix in proxy_get_request.
Root cause
select_first_worker() returns workers[0].url(), which for DPAwareWorker includes the @rank suffix (kept for registry identification; base_url() is the clean variant). proxy_get_request builds the outbound URL from the raw value.
Suggested fix
Parse the worker URL before building the request, and set the DP rank header the same way the chat path does:
let (base_url, dp_rank) = dp_utils::parse_worker_url(&worker_url);
let url = format!("{}/{}", base_url, endpoint);
let mut request_builder = dp_utils::add_dp_rank_header(self.client.get(&url), dp_rank);
Add a regression test for DP=2: GET /v1/models must return 200 through the router.
Title:
[Bug] DP intra-node GET proxies fail when worker URLs contain @rank (userinfo misinterpretation)Description
When running with intra-node data parallel size > 1, worker registry URLs carry an
@ranksuffix (e.g.http://127.0.0.1:18100@0). The GET proxy path inproxy_get_request(src/routers/http/router.rs) forwards these URLs without stripping@rank:Per RFC 3986,
@in the authority part is parsed as userinfo, so reqwest sends this request to host0instead of127.0.0.1:18100. The request fails and the router returns 500.Affected endpoints (all via
proxy_get_request)GET /v1/models(breaks Swarm / AgentBench preflight)GET /health_generateGET /get_server_infoGET /get_model_infoUnaffected:
POST /v1/chat/completions(already strips@rankviadp_utils::parse_worker_urland sendsX-data-parallel-rank) andGET /health(send_health_checkalready strips viaextract_dp_rank). DP=1 is unaffected since there is no@ranksuffix.Reproduction (uses the repo's own mock worker; verified with the release binary)
Observed output:
The 500 body shows the outbound URL became
http://0.0.0.0/v1/models— the router sent the request to host0(port 80) instead of127.0.0.1:23340; the worker never receives it. POST and DP=1 paths succeed, isolating the bug to the@ranksuffix inproxy_get_request.Root cause
select_first_worker()returnsworkers[0].url(), which forDPAwareWorkerincludes the@ranksuffix (kept for registry identification;base_url()is the clean variant).proxy_get_requestbuilds the outbound URL from the raw value.Suggested fix
Parse the worker URL before building the request, and set the DP rank header the same way the chat path does:
Add a regression test for DP=2:
GET /v1/modelsmust return 200 through the router.