Skip to content

feat(mcp): add per-tool risk annotations#1098

Open
ahjota wants to merge 3 commits into
rohitg00:mainfrom
ahjota:aj/factory-droid-mcp-granular-risks
Open

feat(mcp): add per-tool risk annotations#1098
ahjota wants to merge 3 commits into
rohitg00:mainfrom
ahjota:aj/factory-droid-mcp-granular-risks

Conversation

@ahjota

@ahjota ahjota commented Jul 22, 2026

Copy link
Copy Markdown

What

Adds MCP-standard annotations (readOnlyHint / destructiveHint) to all 53 tools in src/mcp/tools-registry.ts so that MCP clients (notably Droid, via its Autonomy Level risk classification) can auto-run read-only agentmemory tools in low and medium autonomy modes, instead of defaulting every tool to high risk and prompting on each call.

Why

Droid compares each MCP tool's risk level to the session's Autonomy Level and auto-runs when risk is at or below that level (docs).

Tools missing safety metadata are classified as high, so they prompt at every level below Auto (High). Today every agentmemory tool — even pure reads like memory_recall, memory_sessions, memory_slot_list — prompts under Auto (Low) and Auto (Medium), making low/medium autonomy unusable with agentmemory.

With per-tool annotations, the classification becomes:

Risk tier Tools Hint
read-only (23) search, recall, list, get, query, export, audit, etc. { readOnlyHint: true, destructiveHint: false }
non-destructive writes (26) save, consolidate, actions, signals, slots create/append/replace, etc. { readOnlyHint: false, destructiveHint: false }
destructive (4) memory_compress_file, memory_governance_delete, memory_heal, memory_slot_delete { readOnlyHint: false, destructiveHint: true }

Behavioral matrix:

Autonomy Read-only (23) Non-destructive writes (26) Destructive (4)
Off prompt prompt prompt
Low auto-run prompt prompt
Medium auto-run auto-run prompt
High auto-run auto-run auto-run

Auto (High) is unchanged. The change is purely additive metadata.

How

  • src/mcp/tools-registry.ts: extended McpToolDef with an optional McpToolAnnotations field and added annotations to every tool definition. (I admit I let Droid categorize each definition for me.)
  • test/mcp-tool-annotations.test.ts: new behavior-named test asserting the 23/26/4 classification, the mutual-exclusivity of read-only/destructive hints, getVisibleTools() annotation preservation under AGENTMEMORY_TOOLS=all, and that handleToolsList() (standalone tools/list response) carries the same annotations as the registry.
  • AGENTS.md: one-line note on the "adding/removing MCP tools" consistency checklist so future tools carry annotations.

The annotations propagate through the existing tools/list response in both transports (proxy mcp::tools::list returns getVisibleTools(); standalone handleToolsList returns getAllTools()/local fallback), so no server.ts, standalone.ts, transport.ts, or src/triggers/api.ts changes are needed.

How to verify

  • npm install (Node >=20), npm run build — compiles clean.
  • npm test — new test passes; the only failing tests in this worktree are pre-existing and unrelated (test/hook-project.test.ts asserts cwd basename agentmemory but the worktree dir differs; test/embedding-provider.test.ts is env-dependent).
  • Wire-shape: node dist/standalone.mjs in local-fallback mode, issue tools/list, confirm memory_recall reports annotations: { readOnlyHint: true, destructiveHint: false }.
  • End-to-end (manual, in Droid): at Auto (Low) memory_recall auto-runs while memory_save / memory_slot_delete prompt; at Auto (Medium) read-only and non-destructive writes auto-run while destructive still prompts; at Auto (High) everything auto-runs (base case preserved).

Before

I have some nice screenvids of the before and after, but they are too big for GitHub. :) With most recent release of agentmemory, and in a medium autonomy Droid session, I'm testing the memory_export tool call:

image

Note that Droid calls this a high impact tool.

image

(It does this for all of the agentmemory tools.)

After

After running my build of agentmemory and in a low autonomy Droid session:

image

Note that Droid now classifies memory_export as low impact. I change autonomy to medium, which lifts the safety floor:

image

I am not prompted:

image

Closes #1097.

Summary by CodeRabbit

  • New Features

    • Added risk/behavior annotations to every MCP tool, distinguishing read-only from destructive actions.
    • Ensured annotations remain available when retrieving tools across different visibility modes and in the tools list response.
  • Tests

    • Added Vitest coverage to verify every tool has annotations, correct read-only/destructive classification, and exact preservation through tool-list wiring.
  • Documentation

    • Updated MCP maintenance consistency rules to require annotations for each tool entry (including read-only vs destructive hints).

@vercel

vercel Bot commented Jul 22, 2026

Copy link
Copy Markdown

@ahjota is attempting to deploy a commit to the rohitg00's projects Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

MCP tool definitions now support readOnlyHint and destructiveHint annotations. All 53 registered tools are classified, annotations remain available through visible-tool selection and tools/list, and tests validate registry classifications and wire responses.

Changes

MCP risk annotations

Layer / File(s) Summary
Annotation contract and catalog coverage
AGENTS.md, src/mcp/tools-registry.ts
Adds the annotation type and field, classifies all exported MCP tools, and updates maintenance guidance.
Annotation validation and wire preservation
test/mcp-tool-annotations.test.ts
Validates classifications, counts, visibility filtering, and preservation through the tools/list response.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Sequence Diagram(s)

sequenceDiagram
  participant ToolRegistry
  participant getVisibleTools
  participant handleToolsList
  participant MCPClient
  ToolRegistry->>getVisibleTools: provide annotated tool definitions
  getVisibleTools->>handleToolsList: return visible annotated tools
  handleToolsList->>MCPClient: send tools/list response with annotations
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding per-tool risk annotations for MCP tools.
Linked Issues check ✅ Passed The PR adds annotations to all tools, extends the schema, preserves tools/list propagation, and adds the requested coverage and checklist update.
Out of Scope Changes check ✅ Passed The changes stay within the stated scope: registry annotations, tests, and the AGENTS.md consistency note.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/mcp-tool-annotations.test.ts`:
- Around line 149-162: Extend the test around getVisibleTools to also set
AGENTMEMORY_TOOLS to "core" and verify every returned tool has defined
annotations. Preserve the existing environment-variable restoration in the
finally block, and assert the expected filtered result as appropriate for the
core visibility path.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3937830d-e403-4f91-954f-6b0e276aee67

📥 Commits

Reviewing files that changed from the base of the PR and between a8e7d19 and 9d4a27d.

📒 Files selected for processing (3)
  • AGENTS.md
  • src/mcp/tools-registry.ts
  • test/mcp-tool-annotations.test.ts

Comment thread test/mcp-tool-annotations.test.ts Outdated
ahjota added a commit to ahjota/agentmemory that referenced this pull request Jul 22, 2026
Address CodeRabbit feedback on rohitg00#1098: the getVisibleTools test only
exercised AGENTMEMORY_TOOLS=all. Also assert AGENTMEMORY_TOOLS=core
returns the 8 essential tools and that every filtered tool retains
its annotations, so the core code path is covered.

Refs rohitg00#1097

Signed-off-by: AJ Alon <alexander.joel.alon@gmail.com>
ahjota added a commit to ahjota/agentmemory that referenced this pull request Jul 22, 2026
Follow-up to CodeRabbit feedback on rohitg00#1098: hardcoding 53 and 8
duplicated contracts owned elsewhere (tool-count-consistency.test.ts
owns the 53; ESSENTIAL_TOOLS.size is the source of truth for core).
Derive total via getAllTools().length and core via ESSENTIAL_TOOLS.size,
keeping the 23/26/4 classification counts and frozen name sets
hardcoded since those are the contract this test verifies. Adds a
partition sum invariant (23+4+26 === total) that now meaningfully
catches a new tool added without classification.

Refs rohitg00#1097

Signed-off-by: AJ Alon <alexander.joel.alon@gmail.com>
@ahjota

ahjota commented Jul 22, 2026

Copy link
Copy Markdown
Author

Occurs to me that you'd want the full table for all of the MCP tools.

# Tool Tier readOnlyHint destructiveHint Rationale
1 memory_recall read-only true false Searches past observations; pure kv.list read
2 memory_file_history read-only true false Reads observations for given files
3 memory_patterns read-only true false Computes patterns from sessions; no writes (kv.list only)
4 memory_sessions read-only true false Lists recent sessions (kv.list)
5 memory_smart_search read-only true false Hybrid semantic+keyword search
6 memory_vision_search read-only true false CLIP image search; reads stored embeddings
7 memory_timeline read-only true false Chronological read around an anchor
8 memory_export read-only true false Exports memory data as JSON (read + serialize)
9 memory_relations read-only true false Graph traversal read
10 memory_commit_lookup read-only true false Looks up session(s) for a commit SHA
11 memory_commits read-only true false Lists commits linked to sessions
12 memory_graph_query read-only true false BFS graph query; read-only traversal
13 memory_team_feed read-only true false Lists shared team items
14 memory_audit read-only true false Views audit trail
15 memory_frontier read-only true false Lists unblocked actions ranked by priority
16 memory_next read-only true false Returns the single top-ranked next action
17 memory_diagnose read-only true false Runs health checks; reports only, no fixes
18 memory_facet_query read-only true false Queries targets by facet tags
19 memory_verify read-only true false Traces citation chain; read-only provenance
20 memory_lesson_recall read-only true false Searches lessons by query
21 memory_insight_list read-only true false Lists synthesized insights
22 memory_slot_list read-only true false Lists all slots
23 memory_slot_get read-only true false Reads a single slot by label
24 memory_save state-changing false false Writes a new memory
25 memory_profile state-changing false false refresh:true rebuilds and kv.set writes the profile cache
26 memory_claude_bridge_sync state-changing false false Writes/reads MEMORY.md (direction-dependent)
27 memory_consolidate state-changing false false Runs the 4-tier consolidation pipeline (writes)
28 memory_team_share state-changing false false Shares a memory/observation with teammates
29 memory_snapshot_create state-changing false false Creates a git-versioned snapshot
30 memory_action_create state-changing false false Creates an action work item
31 memory_action_update state-changing false false Updates action status/priority/details
32 memory_lease state-changing false false Acquire/release/renew a lease (mutation)
33 memory_routine_run state-changing false false Instantiates a routine, creating actions
34 memory_signal_send state-changing false false Sends/broadcasts a message
35 memory_signal_read state-changing false false Marks delivered messages as read
36 memory_checkpoint state-changing false false create/resolve/list; create+resolve mutate
37 memory_mesh_sync state-changing false false Pushes/pulls to peer instances (replication)
38 memory_sentinel_create state-changing false false Creates an event-driven sentinel
39 memory_sentinel_trigger state-changing false false Fires a sentinel; unblocks gated actions
40 memory_sketch_create state-changing false false Creates an ephemeral action graph
41 memory_sketch_promote state-changing false false Promotes ephemeral actions to permanent
42 memory_crystallize state-changing false false Writes crystal digests via LLM summarization
43 memory_facet_tag state-changing false false Attaches a structured tag to a target
44 memory_lesson_save state-changing false false Saves/strengthens a lesson
45 memory_obsidian_export state-changing false false Writes Markdown files to the vault directory
46 memory_reflect state-changing false false Synthesizes and writes new insights
47 memory_compress_file state-changing false false Overwrites the target file; writes a .original.md backup first, making it reversible
48 memory_slot_create state-changing false false Creates a new slot
49 memory_slot_append state-changing false false Appends text to a slot
50 memory_slot_replace state-changing false false Replaces slot content in place
51 memory_governance_delete destructive false true Deletes memories by ID; audit records what/why but not content, no restore
52 memory_heal destructive false true Auto-fixes issues: kv.delete on orphaned leases, expired signals, sketch actions; no undo
53 memory_slot_delete destructive false true Deletes a slot; permanent, no backup

Totals: 23 read-only + 27 state-changing + 3 destructive = 53.

memory_compress_file was initially classified as destructive, but since the operation automatically creates a .original.md backup before overwriting, it's reversible and correctly classified as state-changing. The three destructive tools all perform permanent kv.delete calls with no restore mechanism.

Mapping to Droid Autonomy Level auto-run behavior:

Level read-only (23) state-changing (27) destructive (3)
Off prompt prompt prompt
Low auto-run prompt prompt
Medium auto-run auto-run prompt
High auto-run auto-run auto-run

@ahjota
ahjota marked this pull request as draft July 22, 2026 07:33
ahjota added 3 commits July 22, 2026 00:40
Add MCP-standard annotations (readOnlyHint/destructiveHint) to all
53 tools so Droid classifies read-only tools as low/medium risk and
auto-runs them under Auto (Low)/Medium, instead of defaulting every
tool to high risk and prompting on each call. Non-destructive writes
are marked non-read-only (medium), destructive tools keep the
destructive hint (high); Auto (High) is unchanged.

Annotations propagate through the existing tools/list response in
both stdio and http transports, so no handler or REST changes are
needed. Tool count is unchanged.

Refs rohitg00#1097

Signed-off-by: AJ Alon <alexander.joel.alon@gmail.com>
Address CodeRabbit feedback on rohitg00#1098: the getVisibleTools test only
exercised AGENTMEMORY_TOOLS=all. Also assert AGENTMEMORY_TOOLS=core
returns the 8 essential tools and that every filtered tool retains
its annotations, so the core code path is covered.

Refs rohitg00#1097

Signed-off-by: AJ Alon <alexander.joel.alon@gmail.com>
Follow-up to CodeRabbit feedback on rohitg00#1098: hardcoding 53 and 8
duplicated contracts owned elsewhere (tool-count-consistency.test.ts
owns the 53; ESSENTIAL_TOOLS.size is the source of truth for core).
Derive total via getAllTools().length and core via ESSENTIAL_TOOLS.size,
keeping the 23/26/4 classification counts and frozen name sets
hardcoded since those are the contract this test verifies. Adds a
partition sum invariant (23+4+26 === total) that now meaningfully
catches a new tool added without classification.

Refs rohitg00#1097

Signed-off-by: AJ Alon <alexander.joel.alon@gmail.com>
@ahjota
ahjota force-pushed the aj/factory-droid-mcp-granular-risks branch from 8de106c to 5485a40 Compare July 22, 2026 07:42
@ahjota
ahjota marked this pull request as ready for review July 22, 2026 07:43

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/mcp-tool-annotations.test.ts`:
- Around line 160-162: Update the annotation assertions in the visible-tools
loop of the mode test to compare each tool’s annotations with the corresponding
registry entry, rather than only checking that annotations are defined. Use the
tool’s identifying name or registry lookup already available in the test, and
preserve the existing coverage across each mode.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c5732a68-597b-445b-85c5-26e87b0cebe7

📥 Commits

Reviewing files that changed from the base of the PR and between 8de106c and 5485a40.

📒 Files selected for processing (3)
  • AGENTS.md
  • src/mcp/tools-registry.ts
  • test/mcp-tool-annotations.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • AGENTS.md
  • src/mcp/tools-registry.ts

Comment thread test/mcp-tool-annotations.test.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Droid: MCP tools force approval at every autonomy level — no risk annotations

1 participant