Skip to content

[Bug]: cached_tokens = prompt_tokens in prefill-decode disagg with stream=true #193

Description

@romain-primalabs

Your current environment

Environment Information
vLLM Router version:
Rust version:
Python version:
Operating System:
Deployment environment:
vLLM backend version:

Component

Prefill/Decode Disaggregation

Router Configuration

vllm-router --policy round_robin \
  --vllm-pd-disaggregation \
  --prefill http://$P:8000 \
  --decode http://$D:8000 \
  --host 0.0.0.0 \
  --port 30000

🐛 Describe the bug

Hello,

I am using vllm-router for prefill-decode disagg where I am using vllm servers with multi connector (lmcache + nixl). It works, but the cached_tokens reported is always the one from decode (getting 100% from prefill server producer).

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions