Skip to content

fix: keep Agent tools when generation_kwargs includes client tools - #12775

Draft
faymarie wants to merge 5 commits into
mainfrom
fix/agent-preserve-tools-with-generation-kwargs
Draft

faymarie wants to merge 5 commits into
mainfrom
fix/agent-preserve-tools-with-generation-kwargs

Conversation

@faymarie

Copy link
Copy Markdown
Contributor

Summary

  • Agent.run(..., generation_kwargs={"tools": [...]}) currently forwards those OpenAI specs into the chat generator. OpenAI chat generators last-write-wins-merge generation_kwargs over the tools argument, so Agent tools disappear from the model.
  • This treats generation_kwargs["tools"] as client-side specs: they are offered to the LLM alongside agent.tools, popped out of generation_kwargs so they cannot overwrite, and never invoked by the Agent.
  • If the model only calls client-side names, the Agent stops with exit_reason="client_tools" and leaves the tool-call message for the caller. If the model also calls an Agent tool, the Agent invokes the owned calls, then stops the same way so the next LLM turn is not missing tool results.

Draft on purpose: this is a proposed Haystack change so mixed Agent + client tool catalogs work (OpenAI-compatible /chat/completions with a server-side Agent). Happy to reshape the API (exit_reason, whether tools belongs in generation_kwargs at all) before this is considered for merge.

Related Issues

  • Issue text is in the accompanying GitHub issue (link after opening).

Proposed Changes

  • Pop tools from run-time generation_kwargs and turn function specs into spec-only Haystack Tool stubs.
  • Pass current_tools + client_side_tools into the chat generator.
  • Invoke only Agent-owned tools. Names that are only in generation_kwargs["tools"] are skipped. Truly unknown names still go through the existing ToolNotFound path.

How did you test it?

Unit tests in test/components/agents/test_agent.py:

  • extra tools are offered next to Agent tools and stripped from generation_kwargs
  • a client-side-only tool call is not invoked (exit_reason=client_tools)
  • an Agent tool is still invoked when client tools are present
  • mixed owned + client calls in one step: invoke owned, then stop

Notes for the reviewer

The generator-level last-write-wins merge in OpenAIChatGenerator._prepare_api_call ({**openai_tools, **generation_kwargs}) is the other half of this bug. This PR fixes the Agent path by not putting tools into generation_kwargs. A generator merge would still help callers that use OpenAIChatGenerator.run(tools=..., generation_kwargs={"tools": ...}) directly, including leftover built-in OpenAI tools that cannot be expressed as Haystack Tool objects.

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have added unit tests and updated the docstrings.
  • I've used a conventional commit type for my PR title.
  • I have added a release note file.
  • I have run ruff and the Agent unit tests locally.

Made with Cursor

generation_kwargs.tools was forwarded to OpenAI chat generators and overwrote Agent.tools. Treat those specs as client-side: offer them to the LLM with Agent tools, invoke only Agent tools, and stop with exit_reason=client_tools so the caller can run the rest.
@vercel

vercel Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
haystack-docs Ignored Ignored Preview Sep 18, 2026 4:06pm UTC

Request Review

@github-actions github-actions Bot added topic:tests type:documentation Improvements on the docs labels Sep 16, 2026
@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/agents
  agent.py 207, 217, 222, 253, 255-256, 294, 398-399
  haystack/components/generators
  utils.py 243, 259, 262-263, 265, 278, 280, 283
  haystack/components/generators/chat
  openai.py
  openai_responses.py 564
Project Total  

This report was generated by python-coverage-comment-action

faymarie and others added 2 commits September 18, 2026 17:33
Share client-tool planning and post-invoke logic between sync and async agent steps, and move OpenAI tool-definition merging into generators/utils to avoid a new chat submodule for schema import.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant