Skip to content

Default enable_thinking off in the agents scaffold for vLLM-served runs - #329

Merged
yilunzhao merged 4 commits into
mainfrom
yilun/scaffold-enable-thinking-default
Aug 27, 2026
Merged

Default enable_thinking off in the agents scaffold for vLLM-served runs#329
yilunzhao merged 4 commits into
mainfrom
yilun/scaffold-enable-thinking-default

Conversation

@yilunzhao

@yilunzhao yilunzhao commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

The openai_agents scaffold builds its own OpenAI client via provider.get_openai_client() and never goes through the provider's generate path, so a provider-level chat_template_kwargs was inert here. The request body this scaffold builds is the only place the value reaches the server.

That matters because no agentic preset configures a vLLM reasoning parser. A backbone whose chat template defaults thinking on would silently run in thinking mode and leak <think> blocks straight into the scored output. The scaffold now sends chat_template_kwargs={"enable_thinking": false} by default, gated to the vLLM-server provider class — the one that exposes a chat_template_kwargs attribute — because managed APIs reject unrecognized body fields with a 400. (A vllm_server provider pointed at a managed base_url would now get the field by default; before, that only happened when the kwarg was set explicitly.) An explicitly configured value still wins; the default only fills a gap.

For a model whose chat template already defaults thinking off, the kwarg is a no-op and nothing changes. This pins an implicit template default explicitly.

Tests: nine cases in tests/core/harness/test_openai_agents_scaffold.py covering the default, an explicit enable_thinking: true winning, unrelated configured kwargs keeping the default, a managed-API provider getting no injection at all, a metrics-wrapped provider still being recognized, the built dict aliasing neither the provider config nor the module-level default, and the logged outcomes. One of them drives the SDK with a stub client to confirm the value lands in the chat completions request body.

The openai_agents scaffold builds its own OpenAI client, so a provider-level
chat_template_kwargs never reached the server on this path. Send it in the
request body instead, defaulting thinking off for providers that support the
field. An explicitly configured value still wins.
@yilunzhao
yilunzhao requested a review from undfined August 26, 2026 21:03

@undfined undfined left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, one suggestion

Comment on lines +63 to +64
if configured is _UNSUPPORTED:
return None

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably want to log something here about this outcome. wdyt?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added — the default now logs once at INFO per worker (the resolver sits behind the agent cache, so it runs once per worker init, not per instance), with the no-op outcomes at DEBUG. Two caplog tests pin the behavior.

Agents are cached per config and provider, so this resolves once per worker:
applying a default is worth INFO, while honoring an explicit value or skipping
a provider that does not accept the field stays at DEBUG.
@yilunzhao
yilunzhao marked this pull request as ready for review August 27, 2026 16:38
@yilunzhao
yilunzhao merged commit c7f14f7 into main Aug 27, 2026
10 checks passed
@yilunzhao
yilunzhao deleted the yilun/scaffold-enable-thinking-default branch August 27, 2026 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants