Skip to content
This repository was archived by the owner on Aug 2, 2026. It is now read-only.

Commit 9ba0bee

Browse files
ptlnextdoorptlnextdoor
committed
docs: rewrite README + add architecture, CLI, daemon-API, troubleshooting
README.md: - Product-first intro with architecture hero image and latency figure. - Quick start: one-line install → provider choice → first search. - CLI walkthrough figure showing the 90% daily flow. - Mermaid source diagram + per-connector table (auth / cadence / credential file / CLI command). - Raycast extension + MCP server sections with install snippets. - Configuration reference (the 8 knobs that matter most). - Resource-limit table (CPU backoff, RAM fallback, idle-aware sync). - 'What's new in v0.3' section summarizing the stability + feature work. docs/architecture.md: - Process model: FastAPI + ThreadPoolExecutor + watchdog + connector scheduler + ChromaDB + embedder + reranker in one process. - Request path walkthroughs for /search, /ingest, connector sync. - Endpoint map classified as fast (constant-time) vs slow (may touch ChromaDB) — explains why /health and /stats are split. - Startup robustness details and storage layout (~/.vef, VEF_DATA_DIR). docs/cli-reference.md: - Every recall / vef-daemon subcommand with flags, example output, and the fig6_cli_walkthrough.png terminal screencap. - Environment variable reference. docs/daemon-api.md: - curl examples for every endpoint with request/response JSON shapes. - Error envelope convention + rate-limit / observability notes. - Liveness vs. readiness distinction explained. docs/troubleshooting.md: - Concrete fixes for the failure modes users actually hit: * 'Daemon process spawned but did not respond within 10s' → ChromaDB corruption recovery steps. * 'Daemon says running but Raycast shows 0 documents' → /stats vs /health mismatch debug path. * hnsw InternalError, EADDRINUSE, huge logs, stale PID, Ollama connection refused, expiring Canvas/Schoology tokens, CPU pinned, 'Python binary not found' from Raycast. * How to capture a diagnostic report. docs/figures/: - fig1_sources.png, fig2_categories.png — index distribution charts. - fig3_latency.png — search latency benchmark. - fig4_architecture.png — system architecture (regenerated, fixes the stale 'Trayce' labels and overlapping text in the previous version). - fig5_indexing_growth.png — corpus growth over time. - fig6_cli_walkthrough.png — simulated terminal showing recall start / status / search / sync / context. AGENT_BUILD_GUIDE.md: - How to add a new connector (subclass Connector, implement sync + is_authenticated, register in __init__, add to setup wizard). - How to add a new MCP tool. - Local dev workflow tips. docs/architecture/local-first-semantic-layer.md: - Design notes for the experimental Rust + WASM runtime scaffold. Made-with: Cursor
1 parent 9c443ce commit 9ba0bee

13 files changed

Lines changed: 2205 additions & 80 deletions

‎AGENT_BUILD_GUIDE.md‎

Lines changed: 1021 additions & 0 deletions
Large diffs are not rendered by default.

‎README.md‎

Lines changed: 354 additions & 80 deletions
Large diffs are not rendered by default.

‎docs/architecture.md‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
# Recall Architecture
2+
3+
Recall is a **local-first semantic memory layer**. A persistent daemon ingests every surface you read and write (local files, Gmail, Google Calendar, Google Drive, Notion, cal.ai, Canvas / Schoology LMS) and folds them into a single ChromaDB collection. Every client — Raycast, the `recall` CLI, and the MCP server for Claude Desktop / Cursor — talks to the same in-process index over a loopback HTTP interface.
4+
5+
![System architecture](figures/fig4_architecture.png)
6+
7+
## Process model
8+
9+
```
10+
┌──────── clients ────────┐
11+
│ │
12+
│ ┌─── Raycast ──────┐ │
13+
│ └─── recall CLI ───┤ │
14+
│ └─── recall-mcp ───┘ │
15+
└───────┬─────────────────┘
16+
│ HTTP 127.0.0.1:19847
17+
▼
18+
sources ───────► ingest queue ──► embedder ──► ChromaDB ──► search + rerank
19+
│ │
20+
├─ filesystem watcher (debounced 2s) │
21+
├─ connector scheduler (idle-aware) │
22+
├─ CPU/RAM guards │
23+
└─ SHA-256 dedup │
24+
▼
25+
~/.vef/chromadb
26+
```
27+
28+
The daemon is **a single Python process** hosting:
29+
30+
| Component | Role |
31+
|---|---|
32+
| FastAPI + Uvicorn | HTTP surface (`/search`, `/health`, `/stats`, `/ingest`, `/sync`, …) |
33+
| `ThreadPoolExecutor` | N=10 ingest workers (`VEF_CONCURRENCY`) |
34+
| `watchdog` observer | Filesystem events per watched dir |
35+
| Connector scheduler | Wakes every 60 s, runs sources whose interval has elapsed |
36+
| ChromaDB client | Cosine-similarity vector store on disk |
37+
| `embedder` module | Gemini / Ollama / NIM provider abstraction |
38+
| `reranker` module | RRF over semantic + BM25 candidates |
39+
40+
## Request paths
41+
42+
### Search (hot path, 30 – 200 ms end-to-end)
43+
44+
```
45+
Raycast ──POST /search──► daemon ──► embedder.embed_query(q)
46+
──► chromadb.query(n=30)
47+
──► reranker.rrf(semantic, bm25)
48+
◄── top-K SearchResult[]
49+
```
50+
51+
`/search` is also used to **debounce connector syncs**: whenever it fires, the scheduler quiets connector work for 30 s so active use is never interrupted.
52+
53+
### Ingest (cold path, async)
54+
55+
```
56+
watchdog event ──► _safe_ingest ──► CPU/RAM guard
57+
──► captioner (images, audio, video)
58+
──► embedder.embed(content_or_caption)
59+
──► store.upsert(sha256, vec, metadata)
60+
```
61+
62+
The captioner uses local Ollama + `faster-whisper` when available — **zero binary payloads leave the machine** unless you explicitly fall back to Gemini binary embedding.
63+
64+
### Connector sync
65+
66+
```
67+
scheduler tick (60 s)
68+
└─ for source in {gmail, gcal, gdrive, calai, canvas, schoology, notion}:
69+
if now - last_sync > interval and not search_in_last_30s:
70+
connector.sync(since=last_sync_token)
71+
for item in new_items:
72+
ingest_item(item, source=source)
73+
```
74+
75+
Each connector persists its own incremental sync token (Gmail historyId, GCal syncToken, etc.) in `~/.vef/credentials/<source>.json`.
76+
77+
## Endpoint map
78+
79+
Two classes of endpoint:
80+
81+
- **Fast (constant-time) endpoints** — safe for liveness probes and UI polling.
82+
- **Slow endpoints** — hit ChromaDB or disk and can take hundreds of ms under load.
83+
84+
| Endpoint | Class | Description |
85+
|---|---|---|
86+
| `GET /health` | fast | Liveness only. Returns `{"status": "ok"}`. Does **not** touch ChromaDB. |
87+
| `GET /stats` | slow | `{status, count}`. Calls `chromadb.count()`. |
88+
| `GET /sources` | fast | Known source tags, sorted. |
89+
| `GET /progress` | slow | In-flight ingest counters + total indexed. |
90+
| `GET /connector-status` | medium | Auth state + last-sync timestamps per connector. |
91+
| `GET /sync-running` | fast | Is a sync currently holding the lock? |
92+
| `GET /watched-dirs` | fast | Configured watcher roots. |
93+
| `POST /search` | hot | Top-K semantic search (optional source filter). |
94+
| `POST /ingest` | slow | Single-file ingest path (used by Raycast "Index this"). |
95+
| `POST /sync` | fast | Kicks a background sync. Returns immediately. |
96+
| `POST /watched-dirs` | fast | Add a watcher root. |
97+
| `DELETE /watched-dirs` | fast | Remove a watcher root. |
98+
| `POST /configure` | fast | Persist API keys into `~/.vef/.env`. |
99+
100+
Full request/response shapes live in [daemon-api.md](daemon-api.md).
101+
102+
## Startup robustness
103+
104+
The daemon CLI (`recall start` / `vef-daemon start`) performs three checks before spawning:
105+
106+
1. **PID-file check** — if `~/.vef/daemon.pid` is valid and the process is alive, reuse it.
107+
2. **Port probe** — if TCP `127.0.0.1:19847` is bound *but* no PID file exists (stale), poll `/health` to confirm liveness.
108+
3. **Spawn with health poll** — fork the uvicorn server, then poll `GET /health` with a 2 s per-attempt timeout until ready.
109+
110+
These checks fix two long-standing races: (a) multiple `recall start` invocations racing for the port and (b) a stale PID file causing the CLI to report "not running" while a healthy daemon is actually bound.
111+
112+
## Storage layout
113+
114+
```
115+
~/.vef/
116+
daemon.pid running daemon PID
117+
daemon.log rotating log (2 MB × 3)
118+
.env persisted API keys (written by /configure)
119+
watched_dirs.json list of folder roots
120+
chroma/ legacy location — wiped on migration
121+
credentials/
122+
gmail.json OAuth refresh tokens (auto-renewed)
123+
gmail_oauth_client.json your Google Cloud Desktop client
124+
gcal.json
125+
gdrive.json
126+
calai.json {"api_key": "cal_..."}
127+
canvas.json {"token": "...", "base_url": "..."}
128+
schoology.json {"consumer_key": "...", "consumer_secret": "..."}
129+
notion.json {"api_key": "ntn_..."}
130+
131+
$VEF_DATA_DIR (default ./data)/
132+
chromadb/ Chroma persistent collection
133+
chroma.sqlite3 meta
134+
<uuid>/ HNSW segment files
135+
```
136+
137+
## Distribution by source and category
138+
139+
From one of the author's personal indexes (1,086 documents):
140+
141+
<p align="center">
142+
<img src="figures/fig1_sources.png" width="45%">
143+
<img src="figures/fig2_categories.png" width="45%">
144+
</p>
145+
146+
Email dominates the corpus because Gmail pulls the last 6 months on first sync; after that the watcher and filesystem drift take over.
147+
148+
## Why this is fast
149+
150+
![Search latency benchmark](figures/fig3_latency.png)
151+
152+
Semantic search over 1,000 embedded documents completes in ~180 ms on an M-series Mac — 4× faster than Spotlight and 25× faster than recursive grep — because:
153+
154+
1. No disk scan. Every document is already a normalized 768-d vector in Chroma's HNSW index.
155+
2. No IPC boundary above loopback HTTP. Raycast's TypeScript runtime talks directly to the in-process FastAPI.
156+
3. `/health` is liveness-only (8 ms) so Raycast can aggressively preflight without triggering false negatives.
157+
158+
## Further reading
159+
160+
- [CLI reference](cli-reference.md) — every `recall …` / `vef-daemon …` subcommand.
161+
- [Daemon HTTP API](daemon-api.md) — request/response shapes with examples.
162+
- [Troubleshooting](troubleshooting.md) — fixes for the failure modes you will actually hit.
163+
- [Local-first semantic layer (Moss)](architecture/local-first-semantic-layer.md) — experimental Rust + WASM runtime.

‎docs/architecture/local-first-semantic-layer.md‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# Local-First Semantic Intelligence Layer (Moss/Trayce) — Architecture v0.1
2+
3+
This document defines a production-grade architecture and implementation scaffold for a local-first semantic runtime that keeps data on-device by default and supports browser, desktop, mobile, and server targets from one Rust core.
4+
5+
## 1) System architecture diagram (ASCII)
6+
7+
```text
8+
┌────────────────────────────┐
9+
│ UI Surfaces │
10+
│ - Spotlight Overlay │
11+
│ - Browser Extension │
12+
│ - Raycast / CLI │
13+
└──────────────┬─────────────┘
14+
│
15+
Local IPC/HTTP (127.0.0.1)
16+
│
17+
┌────────────────────────────┴────────────────────────────┐
18+
│ Runtime Orchestrator │
19+
│ - Query planner (semantic + BM25 + recency) │
20+
│ - Token budget + MMR compression │
21+
│ - Session approvals / injection policy │
22+
└──────────────┬──────────────────────────┬────────────────┘
23+
│ │
24+
Query path │ │ Ingest/sync path
25+
│ │
26+
┌────────────────────▼────────────────┐ ┌────▼────────────────────────┐
27+
│ Embedding Pipeline (local) │ │ Connector Workers │
28+
│ - Text: nomic-e5/minilm (quantized) │ │ Gmail/Drive/GCal/Notion/...│
29+
│ - Vision: moondream/LLaVA (local) │ │ OAuth token via OS keychain│
30+
│ - Audio: Whisper -> text embedding │ │ schedule + incremental sync│
31+
│ - Cache(hash+model+quant) │ └────┬────────────────────────┘
32+
└────────────────────┬──────────────────┘ │
33+
│ │ normalized content
34+
▼ ▼
35+
┌─────────────────────────────────────────────────┐
36+
│ Rust Core Retrieval Engine (shared) │
37+
│ - HNSW ANN + Flat exact fallback │
38+
│ - Namespaces, filters, soft deletes │
39+
│ - search()/searchStream()/compact()/snapshot │
40+
│ - Telemetry + HDR histogram │
41+
└───────────────┬─────────────────────┬───────────┘
42+
│ │
43+
vector index│ │metadata/graph/audit
44+
▼ ▼
45+
┌────────────────┐ ┌────────────────────────────┐
46+
│ Encrypted Store │ │ SQLite + FTS5 + Entity KG │
47+
│ AES-256-GCM │ │ docs, entities, edges, log │
48+
│ mmap/ArrayBuffer│ │ injection audit │
49+
└────────────────┘ └────────────────────────────┘
50+
51+
Targets from same Rust core:
52+
- Browser/Edge: wasm32-unknown-unknown (ESM + Workers)
53+
- Desktop: Tauri backend
54+
- Mobile: UniFFI (Swift/Kotlin)
55+
- Server Node: napi-rs adapter (+ WASM fallback)
56+
- Server Python: PyO3 asyncio adapter
57+
```
58+
59+
## 2) Rust crate layout and responsibilities
60+
61+
```text
62+
moss/
63+
Cargo.toml # workspace root
64+
README.md # quickstarts + target matrix
65+
crates/
66+
moss-core/ # no_std-friendly retrieval and token runtime
67+
src/
68+
lib.rs
69+
config.rs # index/search tuning knobs
70+
error.rs # explicit error surface
71+
types.rs # Vector, Document, QueryResult, metadata
72+
query.rs # SearchOptions, predicates
73+
distance.rs # cosine/dot similarity kernels
74+
cache.rs # embedding cache (hash/model/quant key)
75+
token_budget.rs # MMR + budget planner
76+
telemetry.rs # structured events + histogram
77+
client.rs # MossCore unified API implementation
78+
index/
79+
mod.rs # IndexEngine enum + trait
80+
flat.rs # exact search path (<10k vectors)
81+
hnsw.rs # ANN graph path (incremental upsert)
82+
tests/shared_protocol.rs # protocol-driven integration tests
83+
benches/latency_recall.rs # reproducible benchmark harness
84+
moss-wasm/ # wasm-bindgen bridge for browser/edge
85+
src/lib.rs
86+
moss-bench/ # CLI benchmark runner for CI + local perf checks
87+
src/main.rs
88+
sdk/
89+
browser/ # TypeScript MossClient wrapper over wasm
90+
src/client.ts
91+
src/types.ts
92+
src/index.ts
93+
protocol/
94+
shared-protocol.schema.json # cross-platform test contract
95+
cases/mvp-smoke.json # reference scenario
96+
```
97+
98+
## 3) Non-functional guarantees and guardrails
99+
100+
1. **Privacy-first default**: no network egress during file indexing or querying unless user explicitly enables connector sync or cloud sync.
101+
2. **Security boundary**: index snapshots and metadata stores encrypted with AES-256-GCM; connector tokens in OS keychain only.
102+
3. **Isolation**: namespace-scoped queries by default; cross-namespace queries require explicit opt-in.
103+
4. **Hot-path constraints**: bounded candidate sets (`ef_search`, `top_k`) and no unbounded heap growth in retrieval loops.
104+
5. **Observability**: every search emits latency + scoring telemetry; histogram tracked in-process.
105+
106+
## 4) Delivery split
107+
108+
- **v0.1 (this scaffold):** local indexing primitives, HNSW/flat engine scaffold, browser SDK wrapper, shared protocol tests, benchmark CLI.
109+
- **v0.2:** full connector set, graph canvas, auto context detection in extension, cross-platform packaging (Windows/Linux/mobile), encrypted cloud sync.

0 commit comments

Comments
 (0)