Repository navigation
Commit 1224fef
docs(papers): add longctx-1m-and-triattention + TriAttention V3 §10 addendum
Two new pieces of paper-form documentation covering longctx's two
operating modes and the V3 rescue fix.
NEW: docs/papers/longctx-1m-and-triattention.md
- Long-form paper on longctx — open long-context retrieval companion
for inference engines. Two modes: (1) standalone retrieval scored
on MRCR v2 8-needle on AMD MI300X with Qwen2.5-32B-Instruct, (2)
rescue layer for our TriAttention V3 eviction policy on Apple
Silicon, validated end-to-end via 256K planted-fact NIAH on
Qwen3.5-2B-4bit.
- Headline numbers: MRCR v2 1M MultiQ Selector + bge-rerank + det
copy n=30 directional 0.688 (above SubQ 0.659; mass-val pending).
V3+longctx ✓HIT every rung 32K → 256K via ChatSession;
V3-only ✗miss every rung.
- Architecture diagrams: full-stack (clients → vllm-swift →
ChatSession → mlx-swift-lm → V3 engine + rescue → longctx-svc),
V3 turn-by-turn flow (eviction-write on turn N, rehydrate-retrieve
before turn N+1).
- Appendices: module map (mlx-swift-lm side), longctx-svc endpoint
surface, V3 environment variable reference table.
- Cites Mao et al., "TriAttention: Efficient Long Reasoning with
Trigonometric KV Compression" (arXiv:2604.04921, 2026) as the
original paper. V3 in this stack is our independent Swift port +
hybrid extension; cross-references docs/papers/triattention-v3.md.
UPDATED: docs/papers/triattention-v3.md
- Adds §10 "Addendum: Swift Port + longctx Rescue — Fixing the
Hybrid NIAH Failure" after the §9 vLLM addendum.
- Reports the Apple Silicon Swift port test results that resolve the
Section 7.1 question "why does V3 break on hybrid Mamba+Attention
models?". Same failure mode reproduces on Swift port; longctx
rescue layer flips it.
- Includes the 12-cell receipt table, the V3 turn-by-turn diagram,
the driver-gotcha section explaining why bare container.generate()
doesn't rescue (Tier-3 rehydrate is ChatSession-only per commit
fe1a3b0 in mlx-swift-lm), and what's still not measured (>256K,
larger hybrids, V3+TQ+ stack).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 28e5178 commit 1224fef
2 files changed
Lines changed: 582 additions & 0 deletions
0 commit comments