Skip to content

Commit c6990e4

Browse files
authored
[codex] tighten memory recall, capture, and resume (volcengine#2598)
* Fix Codex memory hook recall noise and stop timeouts * Tighten Codex recall compression output * Add Codex archive resume and capture filtering * Wrap Codex memory injection for capture filtering * Detect Codex recall compressor profile * Refresh Codex compressor profile on startup * fix codex ov credential resolution * [codex] resolve compressor profile via models_cache.json, not codex exec probe SessionStart used to spawn 'codex exec' sequentially against each candidate model to detect which one would respond — up to 3 probes × ~15s timeout each, on every session start, even on resume. The configured-on-startup default made this a guaranteed first-page-load tax of several seconds. Replace the probe with a lookup against codex's own model catalogue (~/.codex/models_cache.json, refreshed by codex CLI's etag-backed fetch). The first candidate whose slug is present wins. SessionStart now goes cache-first: load the persisted profile if any, only resolve on cache miss. The runtime compress path (auto-recall) deletes the cached profile on any compress failure so the next SessionStart re-resolves against the current catalogue. - recall-compressor-profile.mjs: * loadCodexModelsCache(env) reads ~/.codex/models_cache.json; missing cache yields {present:false,slugs:Set()}. * resolveRecallCompressorProfile picks the first available candidate by slug; falls back optimistically to the first candidate when the catalogue is missing. * invalidateRecallCompressorProfileCache() rms the persisted file. * detectRecallCompressorProfile is now cache-first and never spawns. - auto-recall.mjs: runCodexCompressor invalidates the cache on spawn error, timeout, non-zero exit, and read failure (best-effort, no error surface to user). - recall-compressor-profile.test.mjs: 11 unit tests covering catalogue read, candidate selection (with/without configured first), missing catalogue fallback, configured_off path, invalidate, cache-first detect, and re-resolve after invalidate. Notes: - buildCodexExecArgs is still exported so auto-recall can spawn the actual compress run; the change only removes the *probe* spawn, not the compress spawn. - recallCompressDetectTtlMs and recallCompressDetectTimeoutMs are preserved in config for back-compat; the timeout no longer matters but the TTL still bounds how stale a cached profile may be. * [codex] omit X-OpenViking-Actor-Peer env_http_headers when no peer configured syncMcpConfig used to unconditionally write all three OV header→env mappings. The wrapper strips empty OPENVIKING_PEER_ID before exec'ing codex, so an unset env var would silently flip the header to "" — the OV side then has to disambiguate that from "no peer scope". Match the bearer_token_env_var pattern: present only when there's something to send. Also drops a stale X-OpenViking-Actor-Peer entry when the peer is unset (e.g. after switching ovcli configs). - Existing test 4 became two cases: with-peer keeps the mapping, without-peer drops it (symmetric to bearer). - New test asserts an in-place drop when the cached .mcp.json had a stale peer mapping but the active config no longer has a peer. * [codex] runtime_failed compressor marker stops same-session retry storms Previous fix invalidated the profile cache on compress failure. Within a single codex session that still bled `recallCompressTimeoutMs` of wall time per UserPromptSubmit because the next hook reread cache (miss), fell back to fallbackRecallCompressorProfile, and tried the same model. Replace plain invalidate with a runtime_failed sentinel cached in the profile slot. UserPromptSubmit's compressMemoryContext already short- circuits on `profile.enabled === false`, so the marker stops further spawns for the rest of the codex process. The next SessionStart cache- first detect treats `source === 'runtime_failed'` as cache miss and re-resolves against the current models_cache.json, so a transient failure self-recovers across codex restarts without operator action. detect_on_startup=false respects the marker (no auto-recover, matches the "manual control" intent of that flag). - recall-compressor-profile.mjs: * markRecallCompressorRuntimeFailed(cfg, {failedModel}) writes the disabled sentinel. * detectRecallCompressorProfile branches on cached.source === 'runtime_failed': cache hit otherwise, recover-via-resolve when startup-detect on, respect marker when off. - auto-recall.mjs::runCodexCompressor: swap invalidate-on-error with markRecallCompressorRuntimeFailed(cfg, {failedModel: profile.model}). - recall-compressor-profile.test.mjs: 4 new tests covering marker write, cross-restart recovery picking a different slug, and the detect_on_startup=false honor path. 20/20 pass. invalidateRecallCompressorProfileCache is kept as a public API for explicit operator use (e.g. a future `ov codex reset-compressor` command), but is no longer called from the runtime path.
1 parent 794ee3e commit c6990e4

19 files changed

Lines changed: 2320 additions & 464 deletions

‎docs/en/agent-integrations/04-codex.md‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ codex # First run: approve hooks once when prompted via /hooks
3232

3333
Prerequisites: Node.js >= 22, Codex >= 0.130.0, and the `codex_hooks` feature enabled.
3434

35-
1. **Shell function wrapper** — Append a `codex()` function to your shell profile (e.g., `.bashrc` or `.zshrc`) to inject OpenViking environment variables from `ovcli.conf`. See the [plugin README](https://github.com/volcengine/OpenViking/blob/main/examples/codex-memory-plugin/README.md) for the full function.
35+
1. **Shell function wrapper** — Source the plugin wrapper from your shell profile (e.g., `.bashrc` or `.zshrc`) so Codex receives the active `ovcli.conf` credentials, MCP is re-rendered, and stale inherited credential env vars are stripped. See the [plugin README](https://github.com/volcengine/OpenViking/blob/main/examples/codex-memory-plugin/README.md) for the snippet.
3636

3737
2. **Plugin installation** — Register a local marketplace and enable the plugin. See `setup-helper/install.sh` for the exact commands.
3838

@@ -46,7 +46,7 @@ Prerequisites: Node.js >= 22, Codex >= 0.130.0, and the `codex_hooks` feature en
4646
type codex # Expectation: codex is a shell function
4747
```
4848

49-
> If the previous command printed a path instead of `shell function`, the wrapper isn't active yet. Re-source it (or open a new terminal) before launching, otherwise codex starts without `OPENVIKING_API_KEY` and reports `MCP server is not logged in`.
49+
> If the previous command printed a path instead of `shell function`, the wrapper isn't active yet. Re-source it (or open a new terminal) before launching, otherwise Codex may start without the credentials needed by MCP.
5050
5151
Once inside Codex, the plugin should seamlessly recall memories on every prompt. Set `OPENVIKING_DEBUG=1` to write events to `~/.openviking/logs/codex-hooks.log`.
5252

@@ -59,12 +59,14 @@ The plugin integrates with Codex's lifecycle by hooking into key events. It sear
5959
<details>
6060
<summary><b>Configuration</b></summary>
6161

62-
Configuration priority: Environment variables > `ovcli.conf` > `ov.conf` > Built-in defaults (`http://127.0.0.1:1933`, no authentication).
62+
Credential source: active `ovcli.conf` wins by default (`OPENVIKING_CLI_CONFIG_FILE` or `~/.openviking/ovcli.conf`), so `ov config switch <name>` changes hooks, MCP, and child `ov` commands together on the next launch. Set `OPENVIKING_CREDENTIAL_SOURCE=env` only when you intentionally want env vars to override the CLI config; the wrapper then writes a mode-0600 runtime ovcli config under `~/.openviking/codex-plugin-state/` so child `ov` commands still match. Without an ovcli config, env vars then `ov.conf` then built-in defaults are used.
6363

6464
| Env Var | Default | Description |
6565
|---------|---------|-------------|
6666
| `OPENVIKING_URL` / `OPENVIKING_BASE_URL` | — | Full server URL |
6767
| `OPENVIKING_API_KEY` | — | API key (sent as `Authorization: Bearer`) |
68+
| `OPENVIKING_CLI_CONFIG_FILE` | `~/.openviking/ovcli.conf` | Active CLI config to use for hooks, MCP, and child `ov` commands |
69+
| `OPENVIKING_CREDENTIAL_SOURCE` | `auto` | Set `env` to force env-var credentials instead of active ovcli config |
6870
| `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` | `120000` | SessionStart active-window threshold |
6971
| `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` | SessionStart idle-TTL sweep threshold |
7072
| `OPENVIKING_DEBUG` | `false` | Write logs to `~/.openviking/logs/codex-hooks.log` |
@@ -77,13 +79,13 @@ Additional tuning options (e.g., `OPENVIKING_RECALL_LIMIT`, `OPENVIKING_CAPTURE_
7779

7880
| Symptom | Cause | Fix |
7981
|---------|-------|-----|
80-
| `MCP server is not logged in` | `OPENVIKING_API_KEY` not in env at launch | Ensure the `codex()` shell function is sourced and `ovcli.conf` contains a valid `api_key`. |
82+
| `MCP server is not logged in` | Credentials were not injected at Codex launch, or the active ovcli config has no `api_key` for an authenticated server | Ensure the `codex()` shell function is sourced and `ovcli.conf` contains a valid `api_key`. |
8183
| `type codex` shows a path instead of a shell function (wrapper inactive) | The rc wasn't `source`d after install, or you launched from a terminal that didn't load it | Run `source ~/.zshrc` (or `~/.bashrc` on bash), or open a new terminal |
8284
| Launching via an alias (e.g. `cx`) injects no credentials | The alias *name* was listed in `OPENVIKING_CODEX_WRAP_EXTRA` (alias names are skipped), or the alias's target command isn't wrapped | Wrap the command the alias points to, not the alias: `alias cx=codex` needs nothing; for `alias cx=codex-custom`, add `codex-custom` |
8385
| `4 hooks need review` | Security review on first launch | Run `/hooks` within Codex and approve the hooks. |
8486
| `hook (failed) exited with code 1` | Stale placeholders in the plugin cache | Re-run the one-line installer. |
8587
| Recall returns nothing | Server is unreachable or the URL is incorrect | Check the endpoint: `curl "$(jq -r '.url' ~/.openviking/ovcli.conf)/health"` |
86-
| Hook 401 but MCP works (or vice versa) | Mismatch between environment variables and `ovcli.conf` | Hooks re-read `ovcli.conf` on every execution, whereas MCP reads environment variables only at startup. Restart Codex to sync. |
88+
| Hook 401 but MCP works (or vice versa) | Cached MCP config or launch env is stale | Restart Codex through the wrapper. If you intentionally use env credentials, set `OPENVIKING_CREDENTIAL_SOURCE=env`; otherwise use `ov config switch <name>` or `OPENVIKING_CLI_CONFIG_FILE`. |
8789

8890
## See also
8991

‎docs/zh/agent-integrations/04-codex.md‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ codex # 首次启动需进入 /hooks 完成一次审批
3232

3333
前置条件:需安装 Node.js >= 22、Codex >= 0.130.0,并启用 `codex_hooks` 特性。
3434

35-
1. **Shell 函数封装** — 在 shell 的配置文件(如 rc 文件)中追加一个 `codex()` 函数,确保每次调用时都能从 `ovcli.conf` 注入 OpenViking 相关的环境变量。完整的函数代码请参考 [插件 README](https://github.com/volcengine/OpenViking/blob/main/examples/codex-memory-plugin/README.md)。
35+
1. **Shell 函数封装** — 在 shell 的配置文件(如 rc 文件)中 source 插件 wrapper,确保 Codex 使用当前激活的 `ovcli.conf` 凭据,同时刷新 MCP 配置并清掉继承来的过期凭据环境变量。代码片段请参考 [插件 README](https://github.com/volcengine/OpenViking/blob/main/examples/codex-memory-plugin/README.md)。
3636

3737
2. **插件安装** — 注册本地 marketplace 并启用插件。具体执行命令请参见 `setup-helper/install.sh`。
3838

@@ -46,7 +46,7 @@ codex # 首次启动需进入 /hooks 完成一次审批
4646
type codex # 期望输出:codex is a shell function
4747
```
4848

49-
> 若上一步输出的是一个路径而非 `shell function`,说明 wrapper 尚未生效,请先 `source` 那行 wrapper(或新开一个终端)再启动;否则 codex 启动时拿不到 `OPENVIKING_API_KEY`,会报 `MCP server is not logged in`。
49+
> 若上一步输出的是一个路径而非 `shell function`,说明 wrapper 尚未生效,请先 `source` 那行 wrapper(或新开一个终端)再启动;否则 Codex 可能拿不到 MCP 所需凭据。
5050
5151
进入 Codex 后,插件将在每次用户输入前自动召回记忆。若设置环境变量 `OPENVIKING_DEBUG=1`,则会将相关事件日志写入 `~/.openviking/logs/codex-hooks.log`。
5252

@@ -59,12 +59,14 @@ type codex # 期望输出:codex is a shell function
5959
<details>
6060
<summary><b>配置</b></summary>
6161

62-
配置优先级为:环境变量 > `ovcli.conf` > `ov.conf` > 内置默认值(默认 URL 为 `http://127.0.0.1:1933`,无鉴权)。
62+
凭据来源:默认使用当前激活的 `ovcli.conf`(`OPENVIKING_CLI_CONFIG_FILE` 或 `~/.openviking/ovcli.conf`),因此 `ov config switch <name>` 会在下次启动时同时影响 hook、MCP 和 Codex 内部运行的 `ov` 命令。只有明确希望环境变量覆盖 CLI 配置时,才设置 `OPENVIKING_CREDENTIAL_SOURCE=env`;此时 wrapper 会在 `~/.openviking/codex-plugin-state/` 下写入 mode 0600 的运行时 ovcli 配置,让内部 `ov` 命令仍然使用同一套凭据。若没有 ovcli 配置,则依次回退到环境变量、`ov.conf` 和内置默认值。
6363

6464
| 环境变量 | 默认值 | 说明 |
6565
|---------|--------|------|
6666
| `OPENVIKING_URL` / `OPENVIKING_BASE_URL` | — | 完整的服务器 URL |
6767
| `OPENVIKING_API_KEY` | — | API 密钥(将通过 `Authorization: Bearer` 标头发送) |
68+
| `OPENVIKING_CLI_CONFIG_FILE` | `~/.openviking/ovcli.conf` | hook、MCP 和 Codex 内部 `ov` 命令共同使用的当前 CLI 配置 |
69+
| `OPENVIKING_CREDENTIAL_SOURCE` | `auto` | 设置为 `env` 时强制使用环境变量凭据 |
6870
| `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` | `120000` | `SessionStart` 活动窗口阈值(毫秒) |
6971
| `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` | `SessionStart` 闲置 TTL 清理阈值(毫秒) |
7072
| `OPENVIKING_DEBUG` | `false` | 是否将日志写入 `~/.openviking/logs/codex-hooks.log` |
@@ -77,13 +79,13 @@ type codex # 期望输出:codex is a shell function
7779

7880
| 现象 | 可能原因 | 修复方法 |
7981
|------|------|------|
80-
| `MCP server is not logged in` | 启动时环境变量中缺失 `OPENVIKING_API_KEY` | 确认已 source `codex()` 函数,且 `ovcli.conf` 中配置了 `api_key` |
82+
| `MCP server is not logged in` | Codex 启动时未注入凭据,或当前 ovcli 配置没有 authenticated server 所需的 `api_key` | 确认已 source `codex()` 函数,且 `ovcli.conf` 中配置了 `api_key` |
8183
| `type codex` 显示的是路径而非 shell function(wrapper 未生效) | 安装后未 `source` rc,或在未加载该 rc 的终端里启动 | 执行 `source ~/.zshrc`(bash 用 `~/.bashrc`),或新开一个终端窗口 |
8284
| 通过别名(如 `cx`)启动,凭据未注入 | 把别名名填进了 `OPENVIKING_CODEX_WRAP_EXTRA`(别名会被跳过),或别名指向的命令未被封装 | 封装别名指向的真实命令而非别名本身:`alias cx=codex` 无需配置;`alias cx=codex-custom` 则把 `codex-custom` 填入 |
8385
| `4 hooks need review` | 首次启动需要进行安全审批 | 在 Codex 终端内输入 `/hooks` 完成审批 |
8486
| 审批后仍提示 `hook (failed) exited with code 1` | 缓存文件中的占位符未被正确渲染 | 重新执行一次一键安装脚本 |
8587
| 召回结果为空 | 服务器不可达或 URL 配置错误 | 执行 `curl "$(jq -r '.url' ~/.openviking/ovcli.conf)/health"` 检查服务器状态 |
86-
| Hook 报 401 但 MCP 正常可用,或反之 | 环境变量与 `ovcli.conf` 的配置不一致 | Hook 每次触发均会重新读取 `ovcli.conf`,而 MCP 仅在启动时读取环境变量。请修改配置并重启 Codex。 |
88+
| Hook 报 401 但 MCP 正常可用,或反之 | MCP 缓存配置或启动环境已过期 | 通过 wrapper 重启 Codex。若确实要使用环境变量凭据,设置 `OPENVIKING_CREDENTIAL_SOURCE=env`;否则使用 `ov config switch <name>` 或 `OPENVIKING_CLI_CONFIG_FILE`。 |
8789

8890
## 参见
8991

‎examples/codex-memory-plugin/.mcp.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,8 @@
55
"bearer_token_env_var": "OPENVIKING_API_KEY",
66
"env_http_headers": {
77
"X-OpenViking-Account": "OPENVIKING_ACCOUNT",
8-
"X-OpenViking-User": "OPENVIKING_USER"
8+
"X-OpenViking-User": "OPENVIKING_USER",
9+
"X-OpenViking-Actor-Peer": "OPENVIKING_PEER_ID"
910
}
1011
}
1112
}

0 commit comments

Comments
 (0)