Context
PR #95 added native output_config.format support for Anthropic structured output (Claude 3.5 / 3.7 / 4 series). This works on:
- Anthropic Claude API ✅
- AWS Bedrock ✅
It does NOT work on:
Problem
Customers on Vertex AI or pinned to older Claude 3 models will silently get plain text instead of structured JSON when they pass responseFormat. The capability gate warns but doesn't actually return JSON.
Proposed fix
Implement the forced-tool pattern (matches Vercel AI SDK's behavior):
- When
responseFormat.type === "json_schema" AND model lacks native output_config support, inject a synthetic tool whose input_schema is the user's response schema:
tools: [{ name: "json", description: "Respond as JSON.", input_schema: <user schema> }]
tool_choice: { type: "tool", name: "json" }
- In response parsing, extract the tool call's
input and surface it as content so the SDK behaves identically across all paths.
Reference
Vercel's implementation: packages/anthropic/src/anthropic-language-model.ts:336-352 (in their repo). They gate via structuredOutputMode: 'auto' | 'outputFormat' | 'tool'.
Files to touch
packages/llm-sdk/src/adapters/anthropic.ts — add fallback path in buildRequestOptions + tool extraction in complete() / stream parser
packages/llm-sdk/src/providers/anthropic/provider.ts — same in doGenerate / doStream
packages/llm-sdk/src/providers/anthropic/provider.ts — flip older Claude 3 models from jsonMode: false → true once fallback is wired
Context
PR #95 added native
output_config.formatsupport for Anthropic structured output (Claude 3.5 / 3.7 / 4 series). This works on:It does NOT work on:
output_config)claude-3-opus-20240229,claude-3-sonnet-20240229,claude-3-haiku-20240307) — capability flag set tofalsein PR fix(llm-sdk): structured output across providers + max_completion_tokens for o-series/gpt-5 (closes #93) #95Problem
Customers on Vertex AI or pinned to older Claude 3 models will silently get plain text instead of structured JSON when they pass
responseFormat. The capability gate warns but doesn't actually return JSON.Proposed fix
Implement the forced-tool pattern (matches Vercel AI SDK's behavior):
responseFormat.type === "json_schema"AND model lacks nativeoutput_configsupport, inject a synthetic tool whoseinput_schemais the user's response schema:inputand surface it ascontentso the SDK behaves identically across all paths.Reference
Vercel's implementation:
packages/anthropic/src/anthropic-language-model.ts:336-352(in their repo). They gate viastructuredOutputMode: 'auto' | 'outputFormat' | 'tool'.Files to touch
packages/llm-sdk/src/adapters/anthropic.ts— add fallback path inbuildRequestOptions+ tool extraction incomplete()/ stream parserpackages/llm-sdk/src/providers/anthropic/provider.ts— same indoGenerate/doStreampackages/llm-sdk/src/providers/anthropic/provider.ts— flip older Claude 3 models fromjsonMode: false→trueonce fallback is wired