Skip to content

OpenAIChatGenerator: tools in generation_kwargs silently drop the component's own tools #12832

Description

@CharlsOliver-kws

Describe the bug
OpenAIChatGenerator._prepare_api_call assembles the request payload as {**openai_tools, **generation_kwargs}.
Because generation_kwargs is spliced last, a tools key in it replaces the definitions built from the component's
own tools instead of being added to them. A caller who passes extra OpenAI tool specs alongside Haystack tools ends
up advertising only the former, and the tools it passed as tools are silently dropped.

This is independent of Agent: it reproduces on the chat generator's public API.

Error message
No error is raised — the tools simply never reach the model, which then cannot call them.

Expected behavior
Both sets are advertised to the model. For a name that appears in both, the spec given in generation_kwargs should
win, since that is how every other key in generation_kwargs already takes precedence.

To Reproduce

from unittest.mock import patch

from openai.types.chat import ChatCompletion, ChatCompletionMessage
from openai.types.chat.chat_completion import Choice
from openai.types.completion_usage import CompletionUsage

from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import Tool
from haystack.utils.auth import Secret


def local_search(query: str) -> str:
    return f"kb:{query}"


owned = Tool(
    name="local_search",
    description="Search the knowledge base.",
    parameters={"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]},
    function=local_search,
)
client = {
    "type": "function",
    "function": {
        "name": "search_knowledge_files",
        "description": "Client catalog.",
        "parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]},
    },
}

completion = ChatCompletion(
    id="foo",
    model="gpt-4o",
    object="chat.completion",
    created=0,
    choices=[
        Choice(finish_reason="stop", index=0, message=ChatCompletionMessage(role="assistant", content="ok"))
    ],
    usage=CompletionUsage(completion_tokens=1, prompt_tokens=1, total_tokens=2),
)

with patch("openai.resources.chat.completions.Completions.create", return_value=completion) as m:
    OpenAIChatGenerator(api_key=Secret.from_token("k"), model="gpt-4o").run(
        [ChatMessage.from_user("hi")], tools=[owned], generation_kwargs={"tools": [client]}
    )
    print([t["function"]["name"] for t in m.call_args[1]["tools"]])

# prints ['search_knowledge_files'] ; expected ['local_search', 'search_knowledge_files']

Additional context
#12777 reports the same root cause through Agent, and #12775 fixes the Agent path by not putting tools into
generation_kwargs; its author notes there that "a generator merge would still help callers that use
OpenAIChatGenerator.run(tools=..., generation_kwargs={"tools": ...}) directly". This issue is that generator-side
half, which also covers built-in OpenAI tool specs that cannot be expressed as Haystack Tool objects.
OpenAIResponsesChatGenerator splices the two the same way.

FAQ Check

System:

  • OS: Windows 11
  • Haystack version: main (b717d00)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions