Skip to content

Conversation

@JVenberg
Copy link
Contributor

Title

[Fix] Forward anthropic-beta headers to Bedrock and other cross-providers

Relevant issues

Fixes:

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • I have added a screenshot of my new test passing locally
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix

Changes

Problem

The ProviderSpecificHeader mechanism in the LiteLLM proxy was only supporting single provider matching. When users sent requests with anthropic-beta headers (e.g., context-1m-2025-08-07 for extended context windows) through the proxy to AWS Bedrock, the headers were not forwarded because:

  1. Headers were stored with custom_llm_provider="anthropic"
  2. When Bedrock made the actual request, it looked for custom_llm_provider="bedrock"
  3. The string mismatch caused headers to be dropped

This affected any cross-provider header scenarios where the same header needs to work across multiple providers (Anthropic, Bedrock, Vertex AI, etc.).

Solution

Enhanced the ProviderSpecificHeader mechanism to support multiple providers using a comma-separated format:

1. Updated add_provider_specific_headers_to_request() in litellm/proxy/litellm_pre_call_utils.py:

  • Changed storage from single provider to comma-separated list
  • Now stores: custom_llm_provider="anthropic,bedrock,bedrock_converse,vertex_ai"
  • Maintains backward compatibility with single-provider format

2. Updated get_provider_specific_headers() in litellm/litellm_core_utils/get_provider_specific_headers.py:

  • Modified retrieval logic to split by comma and check if current provider is in the list
  • Simplified logic while maintaining existing behavior

3. Added comprehensive test coverage:

  • 6 new unit tests in tests/test_litellm/litellm_core_utils/test_provider_specific_headers.py
  • 1 new integration test in tests/proxy_unit_tests/test_proxy_utils.py
  • Updated existing test to match new multi-provider format
  • Tests cover multi-provider matching, backward compatibility, and edge cases

Test Results

All 8 tests passing (100% success rate):

  • Multi-provider matching scenarios
  • Backward compatibility with single providers
  • Edge cases (spaces, None values, no matches)

Impact

  • Users can now send anthropic-beta headers through the proxy and have them correctly forwarded to Bedrock
  • Solves extended context window scenarios (1M+ tokens)
  • Maintains backward compatibility - existing single-provider configurations continue to work
  • Minimal code changes (2 core files modified)
  • Low risk due to comprehensive test coverage

Files Modified

  • litellm/proxy/litellm_pre_call_utils.py (lines 1187-1207)
  • litellm/litellm_core_utils/get_provider_specific_headers.py (lines 6-29)
  • tests/test_litellm/litellm_core_utils/test_provider_specific_headers.py (new tests added)
  • tests/proxy_unit_tests/test_proxy_utils.py (tests updated and added)

@vercel
Copy link

vercel bot commented Oct 17, 2025

@JVenberg is attempting to deploy a commit to the CLERKIEAI Team on Vercel.

A member of the Team first needs to authorize it.

@JVenberg JVenberg changed the title [Fix] Forward anthropic-beta headers to Bedrock and other cross-providers [Fix] Forward anthropic-beta headers to Bedrock and other anthropic providers Oct 17, 2025
@ishaan-jaff ishaan-jaff changed the base branch from main to litellm_anthropic_beta_forwarding October 18, 2025 22:05
@ishaan-jaff
Copy link
Contributor

I think this is close, happy to merge and make sure it passes

@ishaan-jaff ishaan-jaff merged commit caafc91 into BerriAI:litellm_anthropic_beta_forwarding Oct 18, 2025
4 of 6 checks passed
ishaan-jaff added a commit that referenced this pull request Oct 18, 2025
* [Fix] Forward anthropic-beta headers to Bedrock and other cross-provider scenarios (#15623)

* add_provider_specific_headers_to_request

* fix add_provider_specific_headers_to_request

* test_provider_specific_header_multi_provider

* test_provider_specific_header_in_request

---------

Co-authored-by: Jack Venberg <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants