Skip to content

feat(flywheel): data flywheel capture/clean/reuse + agent tool-result cache - #7911

Open
bfxh wants to merge 4 commits into
esengine:main-v2from
bfxh:feat/flywheel-toolcache
Open

feat(flywheel): data flywheel capture/clean/reuse + agent tool-result cache#7911
bfxh wants to merge 4 commits into
esengine:main-v2from
bfxh:feat/flywheel-toolcache

Conversation

@bfxh

@bfxh bfxh commented Aug 7, 2026 •
edited
Loading

Copy link
Copy Markdown

Summary

数据飞轮（采集→清洗→复用→验证 四环）+ Agent 层工具结果缓存（建模层缓存命中）。

数据飞轮 internal/flywheel

  • ①采集：Sink 事件流快照（genai.event.v1，OTel GenAI 对齐）+ MCPRecorder（mcp.call.v1）+ 每日轮转 Writer（open-write-close，Windows 无句柄残留）
  • ②清洗：Trajectory/Store 结构化 JSONL（task→verify→judge）+ BM25 轨迹检索（复用 internal/retrieval）
  • ③复用：NOTES.md 记忆目录 + failures/ 失败教训回流 + Judge/HeuristicJudge（failed 自动进记忆）
  • ④验证：VerifyRun 真实验证执行器（闭环：验证→打分→失败回流）
  • 接线：boot NewSink 装配（纯 observer）；MCPRecorder 接 plugin.Client

工具结果缓存 internal/agent/toolcache.go

  • 幂等只读工具（read_file/grep/glob/ls/code_index/纯函数白名单）按 sha256(tool + canonicalizeArgs) 缓存，TTL 30s + 512 上限
  • canonicalizeArgs 参数归一化：JSON 键排序 + 剥离动态噪音键（_ts/timestamp/session_id/seed）+ 数字规范化（1 vs 1.0）；非 JSON 保守原样（宁可 miss 不可错）
  • executeBatch 命中短路跳过真实执行；写工具/bash/webfetch 一律不缓存
  • 可观测：CacheDiagnostics.ToolCacheHits/Misses 随 Usage 事件与 provider 前缀命中率并列

docs/DATA_FLYWHEEL.md

四环架构 + 5 组 schema + 技术雷达（P0-P2）+ §7 agent 层缓存命中

Verification

  • agent/flywheel/plugin 三包全绿（toolcache 4 测试：归一化 9 组/命中过期淘汰/stats/白名单契约）
  • 全仓 109 ok（合并上游后基线）

Notes

CI Declarations

Cache-impact: medium - 新增 Agent 层工具结果缓存（ToolCache：白名单幂等工具 + 参数归一化 + TTL30s/512 上限），命中跳过执行；system prompt 前缀结构不变
Cache-guard: internal/agent/toolcache_test.go（命中/过期淘汰/stats/白名单契约）+ 既有 cachehit e2e 前缀稳定测试
System-prompt-review: 无 system prompt 内容改动，仅 boot 装配点（NewSink/MCPRecorder 纯 observer）；tool schema 顺序排序保证前缀稳定
Documentation-impact: updated - docs/DATA_FLYWHEEL.md（四环架构）新增

lbx13 added 2 commits August 8, 2026 05:06
lbx13
feat(flywheel): data flywheel 采集/清洗/复用三层落地
5fa55c8 
- ring1 采集: flywheel.Sink 事件流快照 (genai.event.v1, OTel GenAI 对齐)
  + MCPRecorder (mcp.call.v1) + 每日轮转 Writer (open-write-close 对齐
  stats 模式, 修复 Windows 句柄占用导致的测试清理失败)
- ring2 清洗: Store/Trajectory (task→verify→judge 结构化 JSONL) +
  BM25 轨迹检索 (复用 internal/retrieval)
- ring3 复用: 记忆目录 NOTES.md + failures/<id>.md (失败教训回流) +
  Judge 接口 + HeuristicJudge (failed 自动进记忆)
- 接线: boot.NewSink 装配 (纯 observer); plugin.Client 改用
  flywheel.MCPRecorder (删除重复的 calllog.go)
- docs/DATA_FLYWHEEL.md: 四环架构 + schema + 技术雷达 (P0-P2)

验证: 全仓 108 ok (仅 3 个已知 Windows symlink 环境限制包失败);
cli 回归 (flywheel events 句柄占用) 修复
lbx13
feat(cache+flywheel): data flywheel 采集/清洗/复用 + agent 层工具结果缓存
edd1964 
- internal/flywheel: Sink 事件流快照(genai.event.v1) + MCPRecorder(mcp.call.v1)
  + Writer(open-write-close) + Trajectory/Store/BM25 检索 + Judge/HeuristicJudge
  + VerifyRun 验证执行器 + NOTES.md/failures 记忆目录
- internal/agent/toolcache.go: 幂等只读工具结果缓存(sha256(tool+canonical args),
  TTL 30s/512 上限) + canonicalizeArgs 参数归一化(键排序/剥动态噪音/数字规范化)
  + executeBatch 短路集成(execute_batch.go) + CacheDiagnostics.ToolCacheHits/Misses
- boot/plugin 接线: NewSink 装配 + MCPRecorder 接 plugin.Client
- docs/DATA_FLYWHEEL.md: 四环架构 + schema + 技术雷达 + §7 缓存命中

cherry-pick from feat/long-horizon-navigator (4a82fc4 + 9cdc480)
@bfxh
bfxh requested review from SivanCola and esengine as code owners August 7, 2026 21:08
@github-actions github-actions Bot added agent Core agent loop (internal/agent, internal/control) mcp MCP servers / plugins (internal/plugin, codegraph) config Configuration & setup (internal/config) v2 Go rewrite (1.x) — main-v2 branch, active development labels Aug 7, 2026
bfxh pushed a commit to bfxh/DeepSeek-Reasonix that referenced this pull request Aug 8, 2026
lbx13
docs(modeling): v2 增补——优化面补全 + PR 对接 + 重复建模场景细分
940f1ca 
- §8 优化面补全（不止缓存）：状态建模/上下文组装/流程建模/可观测性/记忆/审批安全
  六面，逐项标注对接 PR（esengine#7911 flywheel-toolcache / esengine#7912 subagent-steer / esengine#7791 OSWorld）
- §9 重复建模场景细分（R1-R6）：任务模板复用/同目标重试/增量修改/跨会话同构/
  表面相似/全新——逐类判定是否值得语义缓存 + 细分判定流程 + 度量标准
  （只统计 R1/R3 命中，误判率>5% 收紧阈值，防刷命中率）
bfxh pushed a commit to bfxh/DeepSeek-Reasonix that referenced this pull request Aug 8, 2026
lbx13
feat(flywheel): Export 轨迹导出接口（Co-training 素材，对接 esengine#7911）
29b7a8b 
- Store.Export(dir, filter): 按 MinScore/Repo/Task/Label 过滤导出标准化 JSONL
  （每轨迹一文件 + manifest.jsonl 摘要行，供训练脚本筛选）
- 只读聚合不改动原轨迹；judge 标签/分数保留
- docs/MODELLING_ENGINEERING.md §5 阶段 B 第一步

验证: TestExportFiltersAndManifest 全绿（全量/标签/分数/任务+repo/缺失过滤/JSON 有效性）
@bfxh bfxh mentioned this pull request Aug 8, 2026
Open
lbx13
sync(flywheel): bring esengine#7911 up to main — Export/Replay/valida…
c56b139 
…teStoredID

esengine#7911 fell behind the dev line: Export (trajectory export + manifest),
Replay (event-sourcing replay), and the store ID whitelist (validateStoredID)
existed on main but not on this branch. File-level sync from
feat/long-horizon-navigator; flywheel tests green, full repo passes.
@bfxh bfxh mentioned this pull request Aug 8, 2026
Open
bfxh pushed a commit to bfxh/DeepSeek-Reasonix that referenced this pull request Aug 8, 2026
lbx13
chore(esengine#7791): strip cosplay + desktop frontend — converge to …
4c5bae4 
…OSWorld core

esengine#7791 boundary cleanup (step 3): remove the two cleanly-separable domains
(cosplay: 15 files — decided to drop; desktop/frontend: 5 PR esengine#7910 files).
boot/agent already decoupled from cosplay via cosplayhook (5675a52), so the
stripped branch compiles; golden re-recorded, code_verify removed from the
boot contract, eventwire skips when frontend is absent. flywheel/modeling/
steer stay as agent infrastructure (same code as esengine#7911/esengine#7967/esengine#7912, PR value
is separate review not physical separation).
bfxh pushed a commit to bfxh/DeepSeek-Reasonix that referenced this pull request Aug 8, 2026
lbx13
chore(esengine#7791): strip cosplay + desktop frontend — converge to …
337b1a8 
…OSWorld core

Merged latest upstream (a720a3a) + main dev line, then removed the two
cleanly-separable domains: cosplay (15 files, decided to drop) and
desktop/frontend (5 PR esengine#7910 files). boot/agent already decoupled from
cosplay via cosplayhook, so the stripped tree compiles; golden re-recorded,
code_verify removed from the boot contract, eventwire skips when frontend is
absent. flywheel/modeling/steer remain as agent infrastructure (same code as
esengine#7911/esengine#7967/esengine#7912 — separate review, not physical separation).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@SivanCola SivanCola Awaiting requested review from SivanCola SivanCola is a code owner
@esengine esengine Awaiting requested review from esengine esengine is a code owner

Assignees

No one assigned

Labels

agent Core agent loop (internal/agent, internal/control) config Configuration & setup (internal/config) mcp MCP servers / plugins (internal/plugin, codegraph) v2 Go rewrite (1.x) — main-v2 branch, active development

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@bfxh