Optimize Pagination with Full-Page Caching to Reduce API Calls

[akolotov]

Currently, our pagination mechanism for MCP tools is inefficient when interacting with the Blockscout API. The process is as follows:

1. The server fetches a full page of data (e.g., 50 items) from the Blockscout API, which does not support client-side page size configuration.
2. To conserve LLM context, the server returns only a small slice of this data (e.g., 10 items) to the client.
3. The remaining items from the fetched page (e.g., 40 items) are discarded.
4. When the client requests the next page of data, the server makes a completely new API call to Blockscout to fetch the next full page.

This leads to significant redundancy, as the same data is often fetched multiple times across several paginated requests, and a large portion of the downloaded data is wasted on each call.

Proposed Solution

Implement an opportunistic, in-memory caching layer to store the full pages of data returned by the Blockscout API. This will allow the server to serve multiple "sub-pages" (slices) from a single cached API response.

The proposed workflow is:

1. Initial Request: When a tool makes a paginated request for the first time (i.e., with no cursor), it fetches the full page from the Blockscout API.
2. Cache Storage: This full page is stored in an in-memory cache with a reasonable Time-to-Live (TTL), such as 10-15 minutes. The key for the cache is generated deterministically on the server.
3. Smart Cursor Generation: The server generates a "smart cursor" that contains all information required for subsequent requests. The cursor itself remains stateless and does not contain direct references to cache keys. It will encode:
   • The full_page_api_params required to re-fetch the data from Blockscout. This information is also used to deterministically regenerate the cache key on the server.
   • The sub_page_index indicating which slice of the full page is being requested.
4. Subsequent Requests: When a request with a smart cursor is received:
   • The server first decodes the cursor and regenerates the cache_key from the tool name and the full_page_api_params.
   • Cache Hit: The server retrieves the full page from the cache using the regenerated cache_key. It then returns the correct slice based on the sub_page_index without making a new API call.
   • Cache Miss: If the cache entry is not found (e.g., TTL expired), the server uses the full_page_api_params from the cursor to re-fetch the data from the Blockscout API, updates the cache, and then serves the requested slice.

Implementation Details

• Cache Key Generation: The server will generate a deterministic cache_key on each request from the tool name and a canonical representation of the full_page_api_params. This key will not be part of the cursor itself.
• Location: The logic should be implemented primarily within blockscout_mcp_server/tools/common.py, modifying functions like create_items_pagination, apply_cursor_to_params, and the cursor encoding/decoding helpers.
• Caching Library: A simple dictionary with manual timestamp checks or a lightweight library like cachetools could be used.

Benefits

• Reduced API Calls: Drastically reduces the number of requests sent to the Blockscout API for paginated data.
• Faster Responses: Subsequent page requests served from the cache will be significantly faster.
• Efficient Data Usage: Fully utilizes the data fetched from the API instead of discarding most of it.

Open Questions

1. Optimal TTL: What is the ideal TTL for cache entries? A default of 10 minutes seems like a reasonable starting point, but should this be configurable?
2. Cache Implementation: Should we use a standard library like cachetools for more robust cache management (e.g., LRU, TTL policies) or implement a simpler custom solution?
3. Configuration: Should this caching mechanism be enabled by default? Should we provide an environment variable to disable it for debugging or specific use cases?