MCP: add search_in_message tool#8
Closed
endolith wants to merge 1 commit into
Closed
Conversation
This was referenced Jun 16, 2026
wesm
pushed a commit
to kenn-io/msgvault
that referenced
this pull request
Jun 18, 2026
Currently the MCP tool retrieves huge amounts of data with each call, which can overwhelm the context window and crash the conversation in a single response. I've been using vibe-coded improvements in my local copy that greatly narrow what is fed back to the LLM, so it can find things more efficiently and quickly and not waste so much tokens/money, and am going to start submitting PRs for them: 1. endolith#6 2. endolith#7 3. endolith#8 4. endolith#9 (These were written by Cursor so I'm not entirely sure how they work or if they're optimal.) This is the first (and it might still need work): --- ## What changed `search_messages` and `list_messages` previously returned bare arrays (or, for vector/hybrid mode, a custom envelope with hits under `messages`). Both tools now return a shared paginated envelope. **Before — default keyword mode:** ```json [ { "id": 1, "subject": "...", "from": [...] } ] ``` **Before — vector/hybrid mode:** ```json { "query": "...", "mode": "hybrid", "returned": 20, "pool_saturated": false, "generation": { ... }, "messages": [ { "id": 1, "subject": "...", "from": [...] } ] } ``` **Before — list_messages:** ```json [ { "id": 1, "subject": "...", "from": [...] } ] ``` **After — all three:** ```json { "data": [ { "id": 1, "subject": "...", "from": [...] } ], "total": 143, "returned": 20, "offset": 0, "has_more": true } ``` Vector/hybrid mode keeps its extra metadata alongside: ```json { "data": [ { "id": 1, "subject": "...", "from": [...] } ], "total": -1, "returned": 20, "offset": 0, "has_more": true, "mode": "hybrid", "pool_saturated": false, "generation": { ... } } ``` ## Fields | Field | Meaning | |---|---| | `data` | The result rows (was top-level array or `messages`) | | `total` | Total matching rows, or `-1` when unknown | | `returned` | Number of rows in this response | | `offset` | Starting position of this page | | `has_more` | Whether another page exists | `total` is exact for keyword search (reported by `SearchFastCount`). It is `-1` for vector/hybrid modes (since the ranking window is bounded and doesn't enumerate all matches), and for the body-FTS fallback path (since no count query exists for full-text search). When `total=-1`, use `has_more` to determine whether to fetch the next page. [This seems kludgy to me; maybe there's a better way.] ## Behavior changes - **Breaking:** responses are no longer bare arrays. Clients must read `.data`. - **Breaking:** vector/hybrid hits move from `.messages` to `.data`. - [or should it have been the other way around? The shared envelope should follow the existing `.messages` convention?] - Vector/hybrid now supports `offset` pagination within the ranking window. Previously `offset > 0` was rejected with a `pagination_unsupported` error. - `list_messages` now detects more pages by fetching one extra row internally rather than running a separate count query. - Search and list results are now capped at 50 rows per call (previously the shared cap was 1000). ## What comes next Subsequent PRs will change `get_message` to a narrower windowed body, and a new `search_in_message` tool. Both use the same `offset`/`returned`/`has_more` vocabulary established here, so LLM clients can read and search large messages without the full body ever filling the context window. Co-authored-by: endolith <endolith@users.noreply.github.com>
cursor
Bot
force-pushed
the
split/pr2-get-message-paging
branch
from
d95e0f5 to
d19db97
Compare
cursor
Bot
force-pushed
the
split/pr3-search-in-message
branch
from
c72b0e7 to
9034a74
Compare
cursor
Bot
force-pushed
the
split/pr2-get-message-paging
branch
from
d19db97 to
0292d76
Compare
cursor
Bot
force-pushed
the
split/pr3-search-in-message
branch
from
9034a74 to
cb62d60
Compare
New tool finds all occurrences of a term in one message body, returning char_offset, line, and a centered snippet per match. Adds extractContextChar helper (used by search results in a follow-up PR) with UTF-8-safe windows. Co-authored-by: endolith <endolith@gmail.com>
endolith
force-pushed
the
split/pr3-search-in-message
branch
from
cb62d60 to
cea81d6
Compare
endolith
force-pushed
the
split/pr2-get-message-paging
branch
from
0292d76 to
7c260cc
Compare
Owner
Author
|
Close in favor of #15 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Adds
search_in_message— finds all occurrences of a term in one message body.Each match returns:
char_offset(byte offset intobody_text)line(1-based line number)snippet(centered context window)Also adds
extractContextCharhelper (tested here; wired into search results in the next PR).Depends on #7 (get_message windowed reading).
How to use
{"id": 42, "query": "resistor"}Use
char_offsetwithget_message center_atto read a larger window around a match.