feat(semantic): support real transformer embedding models via transformers.js (ONNX)

[limpidwei]

Problem

Model2Vec static embeddings are context-free: token ids are gathered from a fixed matrix and mean-pooled. For CJK vaults this degrades to near bag-of-words — Chinese semantics are highly compositional, and the 256-dim PCA static models cannot separate concepts.

Measured on a real 1,244-note Chinese personal vault (probe query 中间人撮合模式 触达差 信任差):

• lexical: correct note ranked Launch foundations: build/publish + structured logging (SHA-31, SHA-30, SHA-41) #1
• semantic / hybrid (M2V_multilingual_output, the model registered in feat(semantic): register multilingual embedding model (M2V_multilingual_output) #326): top-6 all irrelevant (diary notes, course notes, internship summaries); the correct note absent entirely

The static multilingual model from #326 is fine as a lightweight default, but CJK users need a contextualized model — and the loader is hardwired to the Model2Vec format, so there is currently no way to use one.

What this PR does

Adds an async transformer runtime next to the untouched sync Model2Vec path:

• Format detection (looksLikeTransformerModel): a model dir containing onnx/ + config.json (HuggingFace layout, e.g. a local clone of Xenova/bge-small-zh-v1.5) loads via @huggingface/transformers; anything else stays Model2Vec → fully backward compatible.
• AsyncEmbedder seam (semantic/transformer-embedder.ts): mirrors the sync Embedder interface with embed()/embedBatch() returning Promises. Mean pooling + L2 normalize, dtype: q8, allowRemoteModels = false — the zero-network guarantee and out-of-band fetch contract are preserved exactly.
• Semantic supports both runtimes: batched per-note embedding on the build path (one ONNX session per note instead of per chunk); a warmup embed at load pins dim before SemanticStore construction.
• searchAsync dispatched for the search tool: lexical and Model2Vec paths stay synchronous and byte-identical; hybrid routing logic is unchanged (only the query embedding awaits); MaxSim rerank passes through for runtimes without token vectors (by design, documented).
• README + changeset included; new tests cover format detection, the type guard, and the async search path (620 passed / 1 skipped locally).

Live verification (same vault, after swap)

Same probe query, Xenova/bge-small-zh-v1.5 (22.9 MB quantized ONNX, CPU-only):

• semantic Launch foundations: build/publish + structured logging (SHA-31, SHA-30, SHA-41) #1 = the correct note (0.613), Cross-platform watcher (chokidar) + CI matrix (SHA-35) #2 = its summary page (0.557), top-6 all topically related
• First index build: 1,244 notes / 1,916 chunks in 166 s (then cached per content hash); query-time embedding ~140 ms

Notes for review

• @huggingface/transformers is added as a regular dependency but loaded via dynamic import() — Model2Vec-only installs never pay the ONNX Runtime load cost.
• embedQuery() (sync) now throws a descriptive error when the runtime is async; the dispatched path always uses embedQueryAsync.
• Follow-up idea (separate PR, not included): frontmatter-aware result filtering (e.g. default-excluding status: superseded notes) — happy to discuss shape first.

🤖 Generated with Claude Code

[codacy-production]

Not up to standards ⛔

🔴 Issues 2 high

Alerts:
⚠ 2 issues (≤ 0 issues of at least minor severity)

Results:
2 new issues

Category	Results
Security	2 high

View in Codacy

🟢 Metrics 66 complexity

Metric	Results
Complexity	66

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}