GLM-5.1 fails to understand `tool_reference` response type in Anthropic-compatible tool use

[ai7eam-dev]

Environment

• Model: glm-5.1
• API: Anthropic-compatible endpoint (/api/anthropic)
• Client: Claude Code CLI

Description

When using Claude Code's ToolSearch (deferred tool discovery) feature, GLM-5.1 receives a response containing a tool_reference content block but fails to interpret its meaning, concluding that "no tools were found" instead of proceeding to call the referenced tool.

Reproduction

1. Configure GLM-5.1 as the backend model in Claude Code (via Anthropic-compatible API)
2. Trigger an ultracode/multi-agent workflow (e.g., include the keyword "ultracode" in a prompt)
3. GLM-5.1 calls ToolSearch to discover the Workflow tool
4. The tool response returns:

{
  "content": [
    {
      "type": "tool_reference",
      "tool_name": "Workflow"
    }
  ]
}

5. Expected behavior: GLM-5.1 recognizes tool_reference means the tool is available and calls Workflow directly
6. Actual behavior: GLM-5.1 responds "The tool search didn't return any Workflow tool" and stops execution

Comparison with Other Models

Model	Correctly handles tool_reference
Xiaomi Memo (mimo-v2.5-pro)	✅ Yes
Claude Sonnet 4	✅ Yes
GLM-5.1	❌ No

Root Cause

GLM-5.1's training data does not appear to include the tool_reference response type from Anthropic's tool-use protocol. When a tool_result contains a content block with type: "tool_reference", the model should interpret it as: "this tool is now registered and available for direct invocation." Instead, GLM-5.1 treats it as an unrecognized/empty response.

Workaround

Adding explicit guidance about tool_reference in the system prompt enables GLM-5.1 to handle it correctly:

## Understanding tool_reference Response Type

When ToolSearch returns a response containing:
{"type": "tool_reference", "tool_name": "Workflow"}

This means the tool is now available. Call it directly:
Workflow({script: "...", title: "..."})

I verified at the API level that with this additional system prompt context, GLM-5.1 correctly interprets tool_reference and invokes the Workflow tool as expected.

Suggested Fix

Include tool_reference response type understanding in GLM-5.1's fine-tuning / tool-use training data, so the model natively treats {"type": "tool_reference", "tool_name": "..."} as a signal that the named tool is registered and ready to be called directly — no additional system prompt guidance needed.

Related

• Claude Code uses deferred tool discovery to reduce token usage. Tools like Workflow, TeamCreate, etc. are not loaded at session start and are discovered on-demand via ToolSearch.
• The tool_reference type is part of Claude Code's internal tool protocol for signaling that a deferred tool has been loaded.

[ai7eam-dev]

中文说明

环境

• 模型：glm-5.1
• 接口：Anthropic 兼容 API（/api/anthropic）
• 客户端：Claude Code CLI

问题描述

使用 Claude Code 的 ToolSearch（延迟工具发现）功能时，GLM-5.1 调用 ToolSearch 后收到包含 tool_reference 类型的响应，但无法正确理解该响应的含义，认为"未找到工具"，从而无法继续执行 Workflow 等后续工具调用。

复现步骤

1. 在 Claude Code 中配置 GLM-5.1 作为后端模型
2. 触发 ultracode/多代理工作流
3. GLM-5.1 调用 ToolSearch 查找 Workflow 工具
4. API 返回：

{
  "content": [
    {
      "type": "tool_reference",
      "tool_name": "Workflow"
    }
  ]
}

5. 预期行为：GLM-5.1 识别 tool_reference 表示工具已就绪，直接调用 Workflow 工具
6. 实际行为：GLM-5.1 回复 "The tool search didn't return any Workflow tool"，停止执行

根因分析

GLM-5.1 的训练数据中似乎未包含 Anthropic 工具调用协议中 tool_reference 响应类型的理解。当 tool_result 包含 type: "tool_reference" 的内容块时，模型应将其解释为"该工具已注册，可直接调用"。但 GLM-5.1 将其视为无法识别的空响应。

临时解决方案

在系统提示（CLAUDE.md）中添加针对 tool_reference 的解释后，GLM-5.1 即可正确处理：

## Understanding tool_reference Response Type

When ToolSearch returns a response containing:
{"type": "tool_reference", "tool_name": "Workflow"}

This means the tool is now available. Call it directly:
Workflow({script: "...", title: "..."})

已在 API 级别验证，添加此提示后 GLM-5.1 可正确理解并执行 Workflow 调用。

建议

在 GLM-5.1 的微调/工具调用训练数据中加入 tool_reference 响应类型的理解，使模型原生支持将 {"type": "tool_reference", "tool_name": "..."} 作为"工具已注册、可直接调用"的信号处理。

[JaredforReal]

Thanks for your feedback. And sorry for the inconvenience. We will take a closer look at it.
@ai7eam-dev

[stofancy]

Adding a GLM-5.2 data point — same root cause, different failure mode.

Environment: GLM-5.2 via Anthropic-compatible endpoint, Claude Code CLI, tool_search enabled with 105 deferred tools (including core ones like Read/Bash).

Where 5.1 stops, 5.2 gets stuck in an infinite ToolSearch loop:

• 48 tool_use blocks in the session, all 48 are ToolSearch itself — zero actual tool invocations. The model never executes anything.
• It re-searches the same tools repeatedly: select:Bash ×12, select:Read ×5.
• Its own thinking shows it misreads tool_reference (which carries no schema by design) as "loaded but no schema / not ready":
  • "I keep getting 'Tool loaded' but no actual schema"
  • "I'm stuck in a loop. The ToolSearch keeps returning empty results with 'Tool loaded' messages."

So the underlying defect is identical to this issue — GLM doesn't treat {"type":"tool_reference","tool_name":"X"} as a "ready, call it now" signal. 5.1 stops, 5.2 spins. Disabling tool_search (ENABLE_TOOL_SEARCH=false) sidesteps it entirely, and (per the workaround here) injecting a tool_reference explanation into the system prompt also works.

Happy to share the transcript if useful.

[zRzRzRzRzRzRzR]

This problem only occurs with open-source solutions, but not with non-open-source solutions, right? This is helpful for us to troubleshoot together across multiple frameworks.