Bug description
When clients send a system message whose content is an OpenAI-style
array of content parts (e.g. [{"type":"text","text":"..."}]),
vllm-router silently forwards it as an empty string "", so the system
prompt is completely lost. String-format system content and array-format
user content both work correctly.
Root cause
src/protocols/spec.rs, ChatMessage's custom Deserialize "system" branch:
"system" => Ok(ChatMessage::System {
content: value.get("content")
.and_then(|c| c.as_str()) // array -> None
.unwrap_or("") // -> ""
.to_string(),
...
})
The System variant declares content: String, and Value::as_str()
returns None for arrays, so array content falls back to "".
The User variant already models content as the untagged
UserMessageContent (Text | Parts), so user array messages round-trip fine;
System lacks the same polymorphic handling.
Repro
POST /v1/chat/completions with:
{"model":"...","messages":[
{"role":"system","content":[{"type":"text","text":"You are GLM-5.2."}]},
{"role":"user","content":"hi"}
]}
Forwarded request to upstream vLLM has that system message's content === "".
Suggested fix
Make System.content use the same untagged UserMessageContent type as User,
and deserialize via serde_json::from_value, e.g.:
content: value.get("content")
.map(|c| serde_json::from_value(c.clone())
.unwrap_or(UserMessageContent::Text(String::new())))
.unwrap_or(UserMessageContent::Text(String::new())),
Serialization of the untagged enum already handles string vs array.
Env
router version: main @ d2ba586
Bug description
When clients send a system message whose
contentis an OpenAI-stylearray of content parts (e.g. [{"type":"text","text":"..."}]),
vllm-router silently forwards it as an empty string "", so the system
prompt is completely lost. String-format system content and array-format
user content both work correctly.
Root cause
src/protocols/spec.rs, ChatMessage's custom Deserialize "system" branch:
"system" => Ok(ChatMessage::System {
content: value.get("content")
.and_then(|c| c.as_str()) // array -> None
.unwrap_or("") // -> ""
.to_string(),
...
})
The System variant declares
content: String, andValue::as_str()returns None for arrays, so array content falls back to "".
The User variant already models content as the untagged
UserMessageContent(Text | Parts), so user array messages round-trip fine;System lacks the same polymorphic handling.
Repro
POST /v1/chat/completions with:
{"model":"...","messages":[
{"role":"system","content":[{"type":"text","text":"You are GLM-5.2."}]},
{"role":"user","content":"hi"}
]}
Forwarded request to upstream vLLM has that system message's content === "".
Suggested fix
Make System.content use the same untagged UserMessageContent type as User,
and deserialize via serde_json::from_value, e.g.:
content: value.get("content")
.map(|c| serde_json::from_value(c.clone())
.unwrap_or(UserMessageContent::Text(String::new())))
.unwrap_or(UserMessageContent::Text(String::new())),
Serialization of the untagged enum already handles string vs array.
Env
router version: main @ d2ba586