Support tool and resource caching for MCP servers that support change notifications #3560

dsfaccini · 2025-11-25T23:17:33Z

Fixes #3218 (redundant tools/list calls before every model request)

This PR implements smart caching with proper invalidation for MCP server tools and resources:

Only caches when servers advertise capabilities.tools.listChanged / capabilities.resources.listChanged (checks each independently)
Listens for notifications/tools/list_changed and notifications/resources/list_changed to invalidate caches when tools change dynamically
Clears caches on server reconnection to ensure fresh state

Context:
#3227 added naive caching but was reverted via #3228 because it broke MCP servers that change tools dynamically based on state (per @phemmer's feedback).
This implementation follows @DouweM's recommended approach using the MCP protocol's built-in change notification system.

Changes: Updated MCPServer class to track capabilities, cache tools/resources, handle notifications via message_handler, and invalidate caches accordingly.

…ifications

DouweM · 2025-11-26T19:55:08Z

pydantic_ai_slim/pydantic_ai/mcp.py

+            resources=resources_cap is not None,
+            tools=tools_cap is not None,
+            tools_list_changed=bool(tools_cap.listChanged) if tools_cap else False,
+            resources_list_changed=bool(resources_cap.listChanged) if resources_cap else False,


Let's do this for do this for mcp_capabilities.prompts.listChanged as well, even if we don't support listing prompts yet

not sure to what extent beyond adding the field, but I added the same fields and checks as for the other two (resources/tools)

pydantic_ai_slim/pydantic_ai/mcp.py

DouweM · 2025-11-26T19:59:11Z

pydantic_ai_slim/pydantic_ai/mcp.py

+                self._tools_cache_valid = True
+                return result.tools
+            else:
+                # Server doesn't support notifications, always fetch fresh


I think typically, if a server doesn't support notifications, that means the tools will never change, so we should cache.

Although @phemmer said on the other PR, "a flag might be good for broken mcp servers which change tools without notifications", which I agree with: cache_tools set to True by default, and I suppose cache_resources as well.

Then we should always read/write the cache unless that flag is disabled, with no branching on the server capability.

changed to this now

DouweM · 2025-11-28T20:07:58Z

pydantic_ai_slim/pydantic_ai/mcp.py

+        elif isinstance(message, mcp_types.ResourceListChangedNotification):
+            self._cached_resources = None
+        elif isinstance(message, mcp_types.PromptListChangedNotification):
+            self._cached_prompts = None


Let's remove the cached_prompts stuff because we're not actually loading prompts at all yet

DouweM · 2025-11-28T20:10:30Z

pydantic_ai_slim/pydantic_ai/mcp.py

            max_retries: The maximum number of times to retry a tool call.
            elicitation_callback: Callback function to handle elicitation requests from the server.
+            cache_tools: Whether to cache the list of tools.
+            cache_resources: Whether to cache the list of resources.


Let's link to the full description as it contains very crucial information about when to use this and its behavior. You can use the [...][pydantic_ai.mcp.MCPServer.cache_tools] format

DouweM · 2025-11-28T20:12:21Z

tests/test_mcp.py

+# ============================================================================
+
+
+async def test_tools_caching_enabled_by_default(mcp_server: MCPServerStdio) -> None:


Instead of mocking, can we test against a real MCP server, specifically tests.mcp_server like the rest of this file? You could give it a tool that is hidden by default, plus a special enable_hidden_tool tool that will cause the tool to be able and the notification to be sent.

I assume FastMCP 1.0, which we use in that tests.mcp_server, supports something like that

dsfaccini

I included the live test to trigger the notification that enables the hidden tool.

Looking at mcp.py I stumbled upon the NOTE about the support for audio and the TODO in _mcp.py to support audio. The referenced issue in the FastMCP repo has been closed since then so I added the branch to handle audio, and the respective tests for both.

dsfaccini · 2025-11-30T16:26:45Z

tests/mcp_server.py

+    """Enable the hidden tool, triggering a ToolListChangedNotification."""
+    mcp._tool_manager.add_tool(hidden_tool)  # pyright: ignore[reportPrivateUsage]
+    await ctx.session.send_tool_list_changed()
+    return 'Hidden tool enabled'


added a tool to enable the hidden tool

dsfaccini · 2025-11-30T16:27:17Z

tests/test_mcp.py



+def test_map_from_pai_messages_with_binary_content():
+    """Test that map_from_pai_messages correctly converts image and audio content to MCP format."""


tests mcp image and audio support

dsfaccini · 2025-11-30T16:31:02Z

pydantic_ai_slim/pydantic_ai/_mcp.py

                                        type='image',
-                                        data=base64.b64decode(chunk.data).decode(),
+                                        data=base64.b64encode(chunk.data).decode(),
+                                        mimeType=chunk.media_type,


I thought b64decode was wrong here, because we generally use BinaryContent.data to store arbitrary bytes, not base64-encoded bytes. On closer look it's actually right, there are some tests setting BinaryContent.data = base64_bytes, for example, in test_anthropic and test_mistral.

So an idea: should we have a mechanism to avoid this confusion? Like

enforcing BinaryContent.data to be strictly non-base64 and instead having a property that returns it encoded?

would help dumb humans (like me) and llms (currently equivalent to dumb humans) more easily navigagte and extend the functionality in the future.

dsfaccini · 2025-11-30T16:31:56Z

pydantic_ai_slim/pydantic_ai/mcp.py

-            return messages.BinaryContent(
-                data=base64.b64decode(part.data), media_type=part.mimeType
-            )  # pragma: no cover
+            return messages.BinaryContent(data=base64.b64decode(part.data), media_type=part.mimeType)


952 closed modelcontextprotocol/python-sdk#952

dsfaccini · 2025-11-30T16:32:11Z

pydantic_ai_slim/pydantic_ai/_mcp.py

+                                    'user',
+                                    mcp_types.AudioContent(
+                                        type='audio',
+                                        data=base64.b64encode(chunk.data).decode(),


FastMCP supports audio now, see comment further down

dsfaccini · 2025-11-30T16:48:25Z

pydantic_ai_slim/pydantic_ai/mcp.py


 @dataclass(repr=False, kw_only=True)
 class Resource(BaseResource):
    """A resource that can be read from an MCP server.


just a curious question, so far it doesn't look like we're adding any pydantic-ai-functionality, wouldn't it be okay to use the respective mcp_types.* themselves?

dsfaccini · 2025-11-30T16:49:49Z

tests/test_mcp.py

+        tool_names1 = [t.name for t in tools1]
+        assert 'hidden_tool' not in tool_names1
+        assert 'enable_hidden_tool' in tool_names1
+


this is the live test that uses the enable_hidden_tool to show the hidden_tool

add tool and resource caching for mcp servers that support change not…

8c90789

…ifications

DouweM self-assigned this Nov 26, 2025

DouweM requested changes Nov 26, 2025

View reviewed changes

DouweM added the awaiting author revision label Nov 26, 2025

dsfaccini added 3 commits November 27, 2025 14:31

apply review comments

57e94ff

Merge branch 'main' into mcp-list-tools-intelligently

92aaa8b

reorder and add cahced prompts field

fd9ddfb

dsfaccini requested a review from DouweM November 27, 2025 23:03

DouweM requested changes Nov 28, 2025

View reviewed changes

dsfaccini added 3 commits November 28, 2025 20:46

remove cahed prompts and replace mocked tests

86e204f

add live mcp tool change notification and narrow arg types

1d6041f

fix mcp image content b64decode bug - add support for audio

a715fc4

dsfaccini commented Nov 30, 2025

View reviewed changes

dsfaccini added 3 commits November 30, 2025 12:38

decode was right, fix test

970ca03

cover the continuation of the loop branch

c6ac978

cover the continuation of the loop branch

75da1b7

		# ============================================================================


		async def test_tools_caching_enabled_by_default(mcp_server: MCPServerStdio) -> None:



		def test_map_from_pai_messages_with_binary_content():
		"""Test that map_from_pai_messages correctly converts image and audio content to MCP format."""

Support tool and resource caching for MCP servers that support change notifications #3560

Are you sure you want to change the base?

Support tool and resource caching for MCP servers that support change notifications #3560

Conversation

dsfaccini commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsfaccini left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsfaccini Nov 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dsfaccini commented Nov 25, 2025 •

edited

Loading

dsfaccini left a comment •

edited

Loading

dsfaccini Nov 30, 2025 •

edited

Loading