Default enable_thinking off in the agents scaffold for vLLM-served runs - #329
Merged
Conversation
The openai_agents scaffold builds its own OpenAI client, so a provider-level chat_template_kwargs never reached the server on this path. Send it in the request body instead, defaulting thinking off for providers that support the field. An explicitly configured value still wins.
undfined
approved these changes
Aug 27, 2026
undfined
left a comment
Collaborator
There was a problem hiding this comment.
Looks good, one suggestion
Comment on lines
+63
to
+64
| if configured is _UNSUPPORTED: | ||
| return None |
Collaborator
There was a problem hiding this comment.
We probably want to log something here about this outcome. wdyt?
Contributor
Author
There was a problem hiding this comment.
Added — the default now logs once at INFO per worker (the resolver sits behind the agent cache, so it runs once per worker init, not per instance), with the no-op outcomes at DEBUG. Two caplog tests pin the behavior.
Agents are cached per config and provider, so this resolves once per worker: applying a default is worth INFO, while honoring an explicit value or skipping a provider that does not accept the field stays at DEBUG.
yilunzhao
marked this pull request as ready for review
August 27, 2026 16:38
undfined
approved these changes
Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
openai_agentsscaffold builds its own OpenAI client viaprovider.get_openai_client()and never goes through the provider's generate path, so a provider-levelchat_template_kwargswas inert here. The request body this scaffold builds is the only place the value reaches the server.That matters because no agentic preset configures a vLLM reasoning parser. A backbone whose chat template defaults thinking on would silently run in thinking mode and leak
<think>blocks straight into the scored output. The scaffold now sendschat_template_kwargs={"enable_thinking": false}by default, gated to the vLLM-server provider class — the one that exposes achat_template_kwargsattribute — because managed APIs reject unrecognized body fields with a 400. (Avllm_serverprovider pointed at a managedbase_urlwould now get the field by default; before, that only happened when the kwarg was set explicitly.) An explicitly configured value still wins; the default only fills a gap.For a model whose chat template already defaults thinking off, the kwarg is a no-op and nothing changes. This pins an implicit template default explicitly.
Tests: nine cases in
tests/core/harness/test_openai_agents_scaffold.pycovering the default, an explicitenable_thinking: truewinning, unrelated configured kwargs keeping the default, a managed-API provider getting no injection at all, a metrics-wrapped provider still being recognized, the built dict aliasing neither the provider config nor the module-level default, and the logged outcomes. One of them drives the SDK with a stub client to confirm the value lands in the chat completions request body.