Fix Redis serialization error in SlackAlerting outage alerts #15648

Sameerlite · 2025-10-17T13:10:21Z

Title

Fix Redis serialization error in SlackAlerting outage alerts

Relevant issues

Fixes LIT-1291

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
I have added a screenshot of my new test passing locally
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix

Changes

Problem

When Vertex AI requests timeout (408 errors), LiteLLM attempts to cache outage alert data but fails with:

LiteLLM Redis Caching: async set() - Got exception from REDIS Object of type set is not JSON serializable

This occurs because ProviderRegionOutageModel.deployment_ids is a Python set object, which cannot be JSON serialized for Redis storage.

Solution

Added two helper methods to handle set/list conversion:

_prepare_outage_value_for_cache(): Converts sets to lists before caching
_restore_outage_value_from_cache(): Converts lists back to sets after retrieval

Changes Made

Added helper methods in SlackAlerting class for set/list conversion
Updated region_outage_alerts() method to use helper methods for caching
Updated outage_alerts() method to use helper methods for caching

This fixes Claude's models via the Converse API, which should also fix Claude Code.

* fix(router): update model_name_to_deployment_indices on deployment removal When a deployment is deleted, the model_name_to_deployment_indices map was not being updated, causing stale index references. This could lead to incorrect routing behavior when deployments with the same model_name were dynamically removed. Changes: - Update _update_deployment_indices_after_removal to maintain model_name_to_deployment_indices mapping - Remove deleted indices and decrement indices greater than removed index - Clean up empty entries when no deployments remain for a model name - Update test to verify proper index shifting and cleanup behavior * fix(router): remove redundant index building during initialization Remove duplicate index building operations that were causing unnecessary work during router initialization: 1. Removed redundant `_build_model_id_to_deployment_index_map` call in __init__ - `set_model_list` already builds all indices from scratch 2. Removed redundant `_build_model_name_index` call at end of `set_model_list` - the index is already built incrementally via `_create_deployment` -> `_add_model_to_list_and_index_map` Both indices (model_id_to_deployment_index_map and model_name_to_deployment_indices) are properly maintained as lookup indexes through existing helper methods. This change eliminates O(N) duplicate work during initialization without any behavioral changes. The indices continue to be correctly synchronized with model_list on all operations (add/remove/upsert).

…environment (#14929) Co-authored-by: sotazhang <[email protected]>

merge main

* fix(router): update model_name_to_deployment_indices on deployment removal When a deployment is deleted, the model_name_to_deployment_indices map was not being updated, causing stale index references. This could lead to incorrect routing behavior when deployments with the same model_name were dynamically removed. Changes: - Update _update_deployment_indices_after_removal to maintain model_name_to_deployment_indices mapping - Remove deleted indices and decrement indices greater than removed index - Clean up empty entries when no deployments remain for a model name - Update test to verify proper index shifting and cleanup behavior * fix(router): remove redundant index building during initialization Remove duplicate index building operations that were causing unnecessary work during router initialization: 1. Removed redundant `_build_model_id_to_deployment_index_map` call in __init__ - `set_model_list` already builds all indices from scratch 2. Removed redundant `_build_model_name_index` call at end of `set_model_list` - the index is already built incrementally via `_create_deployment` -> `_add_model_to_list_and_index_map` Both indices (model_id_to_deployment_index_map and model_name_to_deployment_indices) are properly maintained as lookup indexes through existing helper methods. This change eliminates O(N) duplicate work during initialization without any behavioral changes. The indices continue to be correctly synchronized with model_list on all operations (add/remove/upsert).

…environment (#14929) Co-authored-by: sotazhang <[email protected]>

Add tiered pricing and cost calculation for xai

…15503)

…_block_repair Add support for thinking blocks and redacted thinking blocks in Anthropic v1/messages API

(feat) Add voyage model integration in sagemaker

Add support for extended thinking in Anthropic's models via Bedrock's Converse API

* docs: fix doc * docs(index.md): bump rc * [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) * fix: add google_routes to llm_api_routes * test: test_virtual_key_llm_api_routes_allows_google_routes * build: bump version * bump: version 1.78.0 → 1.78.1 * add application level encryption in SQS * add application level encryption in SQS --------- Co-authored-by: Krrish Dholakia <[email protected]> Co-authored-by: Ishaan Jaff <[email protected]> Co-authored-by: deepanshu <[email protected]>

…t/completions API with LiteLLM (#15509) * docs: fix doc * docs(index.md): bump rc * [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) * fix: add google_routes to llm_api_routes * test: test_virtual_key_llm_api_routes_allows_google_routes * add AnthropicCitation * fix async_post_call_success_deployment_hook * fix add vector_store_custom_logger to global callbacks * test_e2e_bedrock_knowledgebase_retrieval_with_llm_api_call * async_post_call_success_deployment_hook * add async_post_call_streaming_deployment_hook * async def test_e2e_bedrock_knowledgebase_retrieval_with_llm_api_call_streaming(setup_vector_store_registry): * fix _call_post_streaming_deployment_hook * fix async_post_call_streaming_deployment_hook * test update * docs: Accessing Search Results * docs KB * fix chatUI * fix searchResults * fix onSearchResults * fix kb --------- Co-authored-by: Krrish Dholakia <[email protected]>

* docs: fix doc * docs(index.md): bump rc * [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) * fix: add google_routes to llm_api_routes * test: test_virtual_key_llm_api_routes_allows_google_routes * build: bump version * bump: version 1.78.0 → 1.78.1 * fix: KeyRequestBase * fix rpm_limit_type * fix dynamic rate limits * fix use dynamic limits here * fix _should_enforce_rate_limit * fix _should_enforce_rate_limit * fix counter * test_dynamic_rate_limiting_v3 * use _create_rate_limit_descriptors --------- Co-authored-by: Krrish Dholakia <[email protected]>

Litellm fix mypy ruff errors1

Litellm staging oct

Add Vertex AI Discovery Engine Rerank Support

vercel · 2025-10-17T13:10:27Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
litellm	Ready	Preview	Comment	Oct 17, 2025 1:30pm

CLAassistant · 2025-10-17T13:10:29Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
5 out of 6 committers have signed the CLA.

✅ lcfyi
✅ ishaan-jaff
✅ AlexsanderHamir
✅ Sameerlite
✅ deepanshululla
❌ LoadingZhang
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

ishaan-jaff

change base to merge into

Sameerlite · 2025-10-20T17:57:04Z

Creating new PR because of conflicts #15741

lcfyi and others added 30 commits October 5, 2025 03:36

Implement fix for thinking_blocks and converse API calls

dd7d12e

This fixes Claude's models via the Converse API, which should also fix Claude Code.

Add thinking literal

1c3ec18

Fix mypy issues

310e3b3

Type fix for redacted thinking

d07455b

Add voyage model integration in sagemaker

25769b5

Add tiered pricing and cost calculation for xai

a33c348

Add config file logic

6002880

Use already exiting voyage transformation

8c7d7a3

Use generic cost calculator

d0e26e2

UI new build

cb9b65d

fix(prometheus): Fix Prometheus metric collection in a multi-workers …

6060537

…environment (#14929) Co-authored-by: sotazhang <[email protected]>

Merge pull request #15477 from BerriAI/main

75ef5df

merge main

Merge pull request #15478 from BerriAI/main

c496097

merge main

fix conversion of thinking block

a39d263

Resolve conflicts in generated HTML files

ef0d6f0

fix(prometheus): Fix Prometheus metric collection in a multi-workers …

b49ab39

…environment (#14929) Co-authored-by: sotazhang <[email protected]>

Merge pull request #15368 from BerriAI/litellm_add_above_128k_pricing

7a68c57

Add tiered pricing and cost calculation for xai

Remove penalty params as supported params for gemini preview model (#…

2afae44

…15503)

Merge pull request #15501 from BerriAI/litellm_fix_anthropic_thinking…

939d499

…_block_repair Add support for thinking blocks and redacted thinking blocks in Anthropic v1/messages API

refactor code as per comments

6fadfd3

fix merge error

c748321

refactor code as per comments

0906b99

refactor code as per comments

a6ed454

Merge branch 'litellm_staging_oct' into litellm_voyage_sagemaker

2653d6a

Merge pull request #15367 from BerriAI/litellm_voyage_sagemaker

2bd7c88

(feat) Add voyage model integration in sagemaker

Merge branch 'litellm_staging_oct' into lcfyi/add-support-for-thinking

cee4b58

Merge pull request #15220 from lcfyi/lcfyi/add-support-for-thinking

0175c13

Add support for extended thinking in Anthropic's models via Bedrock's Converse API

ishaan-jaff and others added 11 commits October 13, 2025 20:05

Add google rerank endpoint

6dd9062

Add docs

2130463

fix mypy error

2362dce

fix mypy and lint errors

4339f52

Merge pull request #15533 from BerriAI/litellm_fix_mypy_ruff_errors1

298fa0c

Litellm fix mypy ruff errors1

Merge pull request #15534 from BerriAI/litellm_staging_oct

43d37a6

Litellm staging oct

Merge pull request #15532 from BerriAI/litellm_google_rerank_endpoint

9af054b

Add Vertex AI Discovery Engine Rerank Support

fix json serializable error for alerts

4a5466d

Add test

2b1b759

Sameerlite changed the base branch from main to litellm_staging_oct October 17, 2025 13:11

fix mypt errors

bbf4759

vercel bot deployed to Preview October 17, 2025 13:20 View deployment

fix mypt errors

378b088

vercel bot deployed to Preview October 17, 2025 13:30 View deployment

Sameerlite force-pushed the litellm_staging_oct branch from 789eec0 to 5a637cc Compare October 17, 2025 18:44

Sameerlite changed the base branch from litellm_staging_oct to main October 17, 2025 18:50

Sameerlite changed the base branch from main to litellm_staging_oct October 17, 2025 18:50

ishaan-jaff requested changes Oct 20, 2025

View reviewed changes

Sameerlite changed the base branch from litellm_staging_oct to litellm_sameer_oct_staging October 20, 2025 17:45

Sameerlite mentioned this pull request Oct 20, 2025

Litellm fix json serialize alreting 2 #15741

Merged

4 tasks

Sameerlite closed this Oct 20, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Fix Redis serialization error in SlackAlerting outage alerts #15648

Fix Redis serialization error in SlackAlerting outage alerts #15648

Uh oh!

Sameerlite commented Oct 17, 2025 •

edited

Loading

Uh oh!

vercel bot commented Oct 17, 2025 •

edited

Loading

Uh oh!

CLAassistant commented Oct 17, 2025 •

edited

Loading

Uh oh!

ishaan-jaff left a comment

Uh oh!

Sameerlite commented Oct 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Uh oh!

Fix Redis serialization error in SlackAlerting outage alerts #15648

Fix Redis serialization error in SlackAlerting outage alerts #15648

Uh oh!

Conversation

Sameerlite commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Title

Relevant issues

Pre-Submission checklist

Type

Changes

Problem

Solution

Changes Made

Uh oh!

vercel bot commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CLAassistant commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ishaan-jaff left a comment

Choose a reason for hiding this comment

Uh oh!

Sameerlite commented Oct 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Sameerlite commented Oct 17, 2025 •

edited

Loading

vercel bot commented Oct 17, 2025 •

edited

Loading

CLAassistant commented Oct 17, 2025 •

edited

Loading