Skip to content

System message with array-formatted content is silently dropped to empty string when forwarding #201

Description

@wkkuniquegmail

Bug description
When clients send a system message whose content is an OpenAI-style
array of content parts (e.g. [{"type":"text","text":"..."}]),
vllm-router silently forwards it as an empty string "", so the system
prompt is completely lost. String-format system content and array-format
user content both work correctly.

Root cause
src/protocols/spec.rs, ChatMessage's custom Deserialize "system" branch:
"system" => Ok(ChatMessage::System {
content: value.get("content")
.and_then(|c| c.as_str()) // array -> None
.unwrap_or("") // -> ""
.to_string(),
...
})
The System variant declares content: String, and Value::as_str()
returns None for arrays, so array content falls back to "".
The User variant already models content as the untagged
UserMessageContent (Text | Parts), so user array messages round-trip fine;
System lacks the same polymorphic handling.

Repro
POST /v1/chat/completions with:
{"model":"...","messages":[
{"role":"system","content":[{"type":"text","text":"You are GLM-5.2."}]},
{"role":"user","content":"hi"}
]}
Forwarded request to upstream vLLM has that system message's content === "".

Suggested fix
Make System.content use the same untagged UserMessageContent type as User,
and deserialize via serde_json::from_value, e.g.:
content: value.get("content")
.map(|c| serde_json::from_value(c.clone())
.unwrap_or(UserMessageContent::Text(String::new())))
.unwrap_or(UserMessageContent::Text(String::new())),
Serialization of the untagged enum already handles string vs array.

Env
router version: main @ d2ba586

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions