Skip to content

[Bug] DP intra-node GET proxies fail when worker URLs contain @rank #220

Description

@wuhang2014

Title: [Bug] DP intra-node GET proxies fail when worker URLs contain @rank (userinfo misinterpretation)

Description

When running with intra-node data parallel size > 1, worker registry URLs carry an @rank suffix (e.g. http://127.0.0.1:18100@0). The GET proxy path in proxy_get_request (src/routers/http/router.rs) forwards these URLs without stripping @rank:

let url = format!("{}/{}", worker_url, endpoint);   // http://127.0.0.1:18100@0/v1/models

Per RFC 3986, @ in the authority part is parsed as userinfo, so reqwest sends this request to host 0 instead of 127.0.0.1:18100. The request fails and the router returns 500.

Affected endpoints (all via proxy_get_request)

  • GET /v1/models (breaks Swarm / AgentBench preflight)
  • GET /health_generate
  • GET /get_server_info
  • GET /get_model_info

Unaffected: POST /v1/chat/completions (already strips @rank via dp_utils::parse_worker_url and sends X-data-parallel-rank) and GET /health (send_health_check already strips via extract_dp_rank). DP=1 is unaffected since there is no @rank suffix.

Reproduction (uses the repo's own mock worker; verified with the release binary)

# 1. Start the repo's mock worker (py_test/fixtures/mock_worker.py)
python py_test/fixtures/mock_worker.py --host 127.0.0.1 --port 23340 > worker.log 2>&1 &

# 2. Start the router with DP=2 (creates workers http://127.0.0.1:23340@0 and @1)
RUST_LOG=vllm_router_rs=debug target/release/vllm-router \
  --host 127.0.0.1 --port 8001 \
  --worker-urls http://127.0.0.1:23340 \
  --policy round_robin \
  --intra-node-data-parallel-size 2 \
  --health-check-interval-secs 1 \
  --prometheus-port 7293 \
  > router.log 2>&1 &

# 3. Wait for "Router ready", then hit the preflight endpoint:
curl -sS -i http://127.0.0.1:8001/v1/models

Observed output:

# router.log
Router ready | workers: ["http://127.0.0.1:23340@0", "http://127.0.0.1:23340@1"]

# GET /v1/models through router (DP=2)
HTTP/1.1 500
Request failed: error sending request for url (http://0.0.0.0/v1/models)

# contrast 1 — direct GET to worker:                       200
curl http://127.0.0.1:23340/v1/models
{"data":[{"id":"mock","object":"model"}]}

# contrast 2 — POST through router (DP=2):                 200
curl -X POST http://127.0.0.1:8001/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"mock","messages":[{"role":"user","content":"hi"}]}'
{"id":"cmpl-...","choices":[{"text":"ok",...}],"worker_id":"worker-23340",...}

# contrast 3 — GET through router with DP=1:               200

The 500 body shows the outbound URL became http://0.0.0.0/v1/models — the router sent the request to host 0 (port 80) instead of 127.0.0.1:23340; the worker never receives it. POST and DP=1 paths succeed, isolating the bug to the @rank suffix in proxy_get_request.

Root cause

select_first_worker() returns workers[0].url(), which for DPAwareWorker includes the @rank suffix (kept for registry identification; base_url() is the clean variant). proxy_get_request builds the outbound URL from the raw value.

Suggested fix

Parse the worker URL before building the request, and set the DP rank header the same way the chat path does:

let (base_url, dp_rank) = dp_utils::parse_worker_url(&worker_url);
let url = format!("{}/{}", base_url, endpoint);
let mut request_builder = dp_utils::add_dp_rank_header(self.client.get(&url), dp_rank);

Add a regression test for DP=2: GET /v1/models must return 200 through the router.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions