Problem
Raven forwards only a fixed subset of sampling parameters to the serving backend. Anything outside that subset is silently dropped, even when the backend supports it. Two incidents where this bit us:
repetition_penalty — sglang supports it, but there is no way to get it through Raven's model config. We had to interpose a local proxy shim on the request path just to set one sampling knob.
enable_thinking: false (Qwen3-style chat-template switch, normally passed via extra_body / chat_template_kwargs) — unreachable through Raven config; required patching the package.
Both are legitimate, backend-documented parameters; needing a fork for each new knob doesn't scale.
Proposal
A passthrough dict on the model/backend config, forwarded verbatim into the request payload (merged into the OpenAI-compatible body / extra_body):
model:
sampling_extra: # merged into sampling params
repetition_penalty: 1.05
request_extra: # merged into extra_body
chat_template_kwargs: { enable_thinking: false }
Exact shape is bikesheddable — the ask is: anything the backend accepts should be reachable from config without a fork.
Problem
Raven forwards only a fixed subset of sampling parameters to the serving backend. Anything outside that subset is silently dropped, even when the backend supports it. Two incidents where this bit us:
repetition_penalty— sglang supports it, but there is no way to get it through Raven's model config. We had to interpose a local proxy shim on the request path just to set one sampling knob.enable_thinking: false(Qwen3-style chat-template switch, normally passed viaextra_body/chat_template_kwargs) — unreachable through Raven config; required patching the package.Both are legitimate, backend-documented parameters; needing a fork for each new knob doesn't scale.
Proposal
A passthrough dict on the model/backend config, forwarded verbatim into the request payload (merged into the OpenAI-compatible body /
extra_body):Exact shape is bikesheddable — the ask is: anything the backend accepts should be reachable from config without a fork.