Upgrade to FastMCP 4 and MCP SDK v2

[zzstoatzz]

Moves the server to FastMCP 4 and the MCP SDK v2. The server code itself needed almost nothing — the interesting work was everywhere else, and most of it was pre-existing breakage that the upgrade dragged into the light.

The mechanical half is what you'd expect: mcp.types became the standalone mcp_types package, and protocol fields went snake_case. FastMCP 4 bridges camelCase reads with a deprecation shim, but it cannot bridge camelCase construction — ToolAnnotations(readOnlyHint=...) silently becomes an unknown kwarg. ty caught those; a grep of the upgrade guide's checklist would not have.

# before — reads warn, construction quietly breaks
from mcp.types import ToolAnnotations
ToolAnnotations(readOnlyHint=True, openWorldHint=False)

# after
from mcp_types import ToolAnnotations
ToolAnnotations(read_only_hint=True, open_world_hint=False)

The evals then needed a second, unrelated migration. pydantic-ai 2.0 dropped MCPServer* and FastMCPToolset for MCPToolset, which takes a transport rather than connection kwargs. That swap is mostly mechanical, but one default flips a lifecycle behavior: StdioTransport keeps the subprocess alive between connections, so a session-scoped fixture caches a session bound to one test's event loop and every later test dies with Event loop is closed. Worth knowing for anyone else making this move.

Session state, checked rather than assumed

PrefectAuthMiddleware stores credentials via ctx.set_state, and sessionless v4 doesn't persist state across requests. This is safe: on_call_tool re-extracts from headers on every call and _prefect_client/client.py reads them back within that same call, so it never crosses a request boundary. The eval logs confirm we're exercising the real 2026-07-28 era — they show MCP send server/discover, not a legacy fallback.

The OAuth provider is RemoteAuthProvider on Cloud JWKS. It doesn't mint tokens, so the v4 issuer_url change and its one-time client re-authorization don't apply here.

Two eval rubrics that were already failing on main

CI treats eval failures as exit 0, so these sat red without a signal. The latest main run was 2 failed, 18 passed; both failures are fixed here, and neither was caused by the upgrade.

test_latest_release_notes fed the judge only release["body"], which omits the release title, then failed the agent for "inventing" it. It also rejected a docs.prefect.io release-notes link as a "non-existent docs page" — that URL returns HTTP 200, and the rubric's own text permits a docs link. The agent was right on both counts. The rubric now passes the title as authoritative material and states that either a docs or GitHub link is acceptable.

test_cloud_oauth_agent_triages_unknown_workspace_failure graded on the exact flow run name while the prompt only asked to identify the workspace and explain the failure. The fixture also creates a similarly-named deployment, so the agent reported that instead. The prompt now asks for the failing run, and the judge is told that also naming the deployment is correct.

Both changes tighten the ask rather than lower the bar — the agent still has to find the right workspace, run, and failure reason. Each was run repeatedly to confirm it's stable, not merely lucky.

Also raised the client init_timeout to 60s on the eval toolsets: each xdist worker cold-starts uv run -m prefect_mcp_server, and importing Prefect under that much parallelism routinely exceeds the 5s default.

Verified with the full suite (124 passed, 2 skipped), evals green 3 runs running, and ruff + ty clean.

This cannot merge yet. pydantic-ai caps fastmcp-slim[client]<4, and the compat work in pydantic/pydantic-ai#6738 deliberately leaves that cap in place. So pyproject.toml currently points at that unmerged branch and carries a uv override lifting the cap — both marked temporary in-file. When #6738 lands and the follow-up widens the range, those come out and this becomes an ordinary version bump. Tracking issue: pydantic/pydantic-ai#6661.

🤖 Generated with Claude Code

[github-actions]

📊 Observability

View eval run traces in Logfire: prefect-mcp-server-evals @ b69bc85

[github-actions]

Evaluation Results

20 tests  ±0   20 ✅ ±0   4m 13s ⏱️ +18s
 1 suites ±0    0 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit b69bc85. ± Comparison against base commit b701a8f.

♻️ This comment has been updated with latest results.