Skip to content

perf(memory): bound the prefix memory index with a recency budget - #8144

Open
chuiwenwei wants to merge 1 commit into
esengine:main-v2from
chuiwenwei:perf/index-prefix-budget
Open

perf(memory): bound the prefix memory index with a recency budget#8144
chuiwenwei wants to merge 1 commit into
esengine:main-v2from
chuiwenwei:perf/index-prefix-budget

Conversation

@chuiwenwei

Copy link
Copy Markdown
Contributor

Summary

  • 现状: MEMORY.md 索引在启动时全量折叠进缓存前缀(system prompt),每轮请求都付这份 token 成本;索引按 Name 排序、陈旧事实永不淘汰(唯一移除路径是手动 forget)——记忆越积越多,前缀成本单调上涨且无上限。
  • 本 PR:新增 IndexBounded ,把进前缀的索引投影改为——陈旧事实(沿用 memoryFreshness:user/feedback 90/365 天、reference 14/45 天、其余 30/180 天)折叠成一行汇总,活跃事实按最近更新降序,1200 字符软预算(至少保留 1 行),折叠行提示 "+N more facts (M stale) — search with the memory tool",告知模型更多记忆存在、可用 memory 工具检索。
  • 磁盘 MEMORY.md 保持全量不变(reindex 照旧),只裁剪进前缀的投影;Index() 全量版保留给工具/诊断。渲染是 (记忆集, now) 的纯函数,会话内前缀字节稳定(cache-first 不变式保持),standing instructions 与召回行为零改动。

Issues

(可留空)

Verification

  • 新增 9 个测试:recency 排序 / stale 折叠(含全 stale)/ 预算裁剪(至少保留 1 行)/ 确定性 / 组合折叠 +N more (M stale) / 零时间戳回落 CreatedAt / 折叠行不匹配 managed 索引正则 / Load 接线预算断言
  • go test ./internal/memory/ ./internal/boot/ 全通过(boot golden 无漂移,testdata 无记忆 fixture)
  • gofmt -l 无输出;go vet 通过
  • 全量 go test -count=1 ./... :仅 2 个已知 macOS sandbox 环境失败(与本 PR 无关,见 fix(sandbox): probe sandbox-exec usability instead of PATH presence fix(sandbox): probe sandbox-exec usability instead of PATH presence #7807

Documentation impact

Documentation-impact: none - 不改记忆模型、命令或配置;索引投影是内部渲染规则,MEMORY.md 的语义描述不变。

Cache impact

Cache-impact: low - 索引投影是 (记忆集, now) 的确定性函数、启动时快照,会话内前缀字节稳定(与现状同一规则);仅投影规则变化(一次性),standing instructions 与 recall 未动。
Cache-guard: 聚焦 guard—— go test ./internal/boot -run TestGoldenBaseline (前缀基线)+ TestIndexBoundedDeterministic (字节稳定)+ 既有 memory 测试;scripts/cache-guard.sh 为 release 级缓存命中检查(本次未跑,改动不触 provider 请求序列化)。
System-prompt-review: esengine — 模型可见的索引列表可能缩减(折叠行保留可发现性,memory 工具仍是检索路径);需维护者确认后合并。

@github-actions github-actions Bot added the v2 Go rewrite (1.x) — main-v2 branch, active development label Aug 10, 2026
@chuiwenwei
chuiwenwei force-pushed the perf/index-prefix-budget branch 2 times, most recently from 3b5ddfe to 7e02c2a Compare August 10, 2026 10:12
The MEMORY.md index folds into the cache-stable system-prompt prefix at
boot, so its size is a per-turn token cost that grows without bound as
facts accumulate: stale facts never leave, and the only removal path is a
manual `forget`.

- IndexBounded renders the prefix projection: stale facts (per
  memoryFreshness) fold into a summary line, the rest order by recency
  (most recently updated first) and cap to a soft 1200-char budget,
  always keeping at least one line. A fold line tells the model more
  memories exist and are reachable via the memory tool.
- The on-disk MEMORY.md stays complete; only the prefix-facing projection
  is capped (Set.Load now uses IndexBounded).
- Rendering is a deterministic function of the memory set, so the prefix
  stays byte-stable within a session (cache-first invariant preserved).

Standing instructions and recall behavior are untouched. Measured
~280 tokens/round on a 10-fact store, with the gain scaling as memories
accumulate.
@chuiwenwei
chuiwenwei force-pushed the perf/index-prefix-budget branch from 7e02c2a to c43feb5 Compare August 11, 2026 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v2 Go rewrite (1.x) — main-v2 branch, active development

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant