Skip to content

Commit 14b684a

Browse files
authored
Merge pull request #37 from alanshurafa/feat/v1.5-codex-build
v1.5: claude-build (Codex orchestrates, Claude executes) + Opus default + Windows status-reader fix
2 parents 215dc83 + deb4669 commit 14b684a

27 files changed

Lines changed: 4164 additions & 72 deletions

.planning/ROADMAP.md

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,19 @@ Co-Evolution is a tooling repo for structured iterative refinement between AI ag
1010
- [x] **v1.1 Polish & Ergonomics** (shipped 2026-04-17) — v1.0 code review fixes (WR-01/02/03) + runtime ergonomics (REVISE auto-loop, visible live mode, branch/worktree management). 4 phases, 6 requirements closed. PR [#2](https://github.com/alanshurafa/co-evolution/pull/2) · See [`milestones/v1.1-ROADMAP.md`](milestones/v1.1-ROADMAP.md) · [`milestones/v1.1-SUMMARY.md`](milestones/v1.1-SUMMARY.md) · [`milestones/v1.1-REQUIREMENTS.md`](milestones/v1.1-REQUIREMENTS.md)
1111
- [x] **v1.3 Reliability, Measurement & Cross-Platform** (shipped 2026-06-11) — stranded-fix landing, macOS/bash-3.2+5.2 portability with 3-OS CI, silent-failure hardening, and the bounce measurement stack (state.json, deterministic scorer + marker-fate ledger, blind judge, human report). Headline: 17.6% deletion-convergence measured; Fable-5 judge 7/7 improved. 9 phases. See [`milestones/v1.3-SUMMARY.md`](milestones/v1.3-SUMMARY.md) · audit at `docs/audits/2026-06-10-v13-audit.md`
1212

13-
## Active Milestone: v1.4 Distribution — npm + MCP (2026-06-11)
13+
## Active Milestone: v1.5 Build with Codex — model ladder + orchestrated execution (2026-06-12)
14+
15+
**Goal:** Adopt the Codex-execution / Fable-orchestration split (per @cjzafir's pattern) in the dev-review runner: fix 3 latent env-export bugs, add per-seat model/effort config, a `--preset codex-build` shortcut, detached background execution with harness-exit-wakeup, a status-reader script, token capture to measure the 50% cost claim, and a `/codex-build` orchestration skill for both the runner and plugin transports. Design basis: `.planning/v1.5-DESIGN.md` (approved 2026-06-12).
16+
17+
- [ ] **Phase 0: Environment + research** (2026-06-12, in progress) — codex symlink + smoke; plugin install; R1 pin `claude -p --output-format json` envelope; R2 pin codex end-of-run token line; register milestone in .planning/; research notes filed.
18+
- [ ] **Phase 1: Seat plumbing + env-export correctness**`lib/co-evolution.sh` effort knobs + `invoke_codex_schema` move (B2); `dev-review.sh` `export CODEX_MODEL` (B1) + `export WORKDIR` (B3) + `--verifier/--claude-model` flags + per-seat env via `apply_seat_env`. Gate: `tests/run-all.sh` green; byte-parity with knobs off.
19+
- [ ] **Phase 2: Claude-verifier hardening + `--preset codex-build`** — fenced-JSON verdict fallback; preset expansion (fable/high → codex/xhigh → fable/max, bounces=2, revise-loop=1); banner; `tests/preset-expansion-simulation.sh`.
20+
- [ ] **Phase 3: Runner observability + status reader**`state.json` additions (`current_phase`, `runner_pid`, `pre/post_execute_sha`, `orchestration.parent_run_id`); new `dev-review-status.sh` (~120 lines, exit codes 0/2/3/4/5); `tests/status-reader-simulation.sh`.
21+
- [ ] **Phase 4: Token capture**`CO_EVOLVE_TOKEN_CAPTURE=1` (default off); `invoke_claude` gated JSON mode; codex stderr harvest; `collect_token_usage``state.json.tokens`; `tests/token-capture-simulation.sh`.
22+
- [ ] **Phase 5: `/codex-build` skill + docs** — new `skills/codex-build/SKILL.md` (preflight → plan → kick → wake/gate loop, both runner and plugin transports); CLAUDE.md Default Rule update; routing doc updates.
23+
- [ ] **Phase 6: Dogfood + evidence** — 2–3 real `/codex-build` tasks (ACCEPT / REVISE→ACCEPT / ESCALATE); token evidence note; MCP parity (`vendor.sh` + `npm test`); memory update.
24+
25+
## Previous Milestone: v1.4 Distribution — npm + MCP (2026-06-11)
1426

1527
**Goal:** Make the bounce protocol invocable without `git clone`: a Node/TS
1628
MCP server (`@alanshurafa/co-evolution-mcp`, one `co_evolve` tool) published

.planning/STATE.md

Lines changed: 22 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,17 @@
11
---
22
gsd_state_version: 1.0
3-
milestone: v1.4
4-
milestone_name: Distribution — npm + MCP
3+
milestone: v1.5
4+
milestone_name: Build with Codex — model ladder + orchestrated execution
55
status: executing
6-
stopped_at: v1.4 Phases 0-4 EXECUTED (2026-06-11). mcp/ package built + 4/4 hermetic smoke tests + stdio handshake verified; CI gains 3-OS mcp job; publish-mcp.yml ready. Remaining = Phase 5 (HUMAN: verify npm scope @alanshurafa, add NPM_TOKEN secret, git tag -> auto-publish, Claude Desktop round-trip) + Phase 6 post-ship registry/awesome-list.
7-
last_updated: "2026-06-10T23:45:00.000Z"
8-
last_activity: 2026-06-11 -- v1.4 milestone registered; v1.3 archived
6+
stopped_at: v1.5 Phases 0-5 EXECUTED (b672145..30a9a00). Phase 6 PARTIAL (2026-06-12) — MCP vendor parity green; codex-verifier degrade path now FULLY GREEN end-to-end: model-leak fix + review-verdict.json schema 400 fix both landed, first real /codex-build ACCEPT produced (subtract-helper task, exit 0, APPROVED conf 96, verify tokens captured). Remaining Phase 6: claude /login (human, for full-ladder + non-zero claude_* tokens), REVISE→ACCEPT row, interactive baseline. v1.4 Phase 5 BLOCKED ON HUMAN (npm scope + NPM_TOKEN + git tag) — running in parallel, untouched.
7+
last_updated: "2026-06-12T17:43:00.000Z"
8+
last_activity: 2026-06-12 -- v1.5 Phase 6: review-verdict.json schema 400 fixed; codex-verifier degrade path E2E green (first ACCEPT)
99
progress:
10-
total_phases: 8
11-
completed_phases: 8
12-
total_plans: 17
13-
completed_plans: 18
14-
percent: 100
10+
total_phases: 7
11+
completed_phases: 6
12+
total_plans: 0
13+
completed_plans: 0
14+
percent: 86
1515
---
1616

1717
# Project State
@@ -30,7 +30,18 @@ Milestone: v1.4 Distribution — npm + MCP
3030
Phase: 0-4 complete; Phase 5 (publish) blocked on human items
3131
Status: npm scope verification + NPM_TOKEN secret + git tag are Alan's; everything else built and CI-gated. v1.2 SC-4 gate still open (VERIFY-SC4.md).
3232
Last activity: 2026-06-10 -- Phase 0 merges + LF policy + audit report
33-
Working directories: `~/co-evolution-v13/` on the Mac (per-machine clone; SMB checkout `/Volumes/Project/co-evolution` is sync-only), `C:/Users/alan/Project/co-evolution-*` on the PC
33+
34+
Milestone: v1.5 Build with Codex — model ladder + orchestrated execution
35+
Phase: 0-5 EXECUTED; Phase 6 (dogfood + evidence) PARTIAL
36+
Status: Phases 0-5 shipped on feat/v1.5-codex-build (b672145 Phase 0 env + research · fb6862a Phase 1 seats + B1/B2/B3 fixes · ffe765f Phase 2 verifier hardening + preset · 13c2bee Phase 3 observability + status reader · fb965ad Phase 4 token capture · 30a9a00 Phase 5 /codex-build skill + docs). Phase 6 partial — see below.
37+
Phase 6 progress (2026-06-12):
38+
- MCP vendor parity GREEN: `bash mcp/scripts/vendor.sh` clean; `(cd mcp && npm test)` 4/4 pass. `mcp/vendor/` is gitignored (generated-at-publish via `npm run build:vendor`, NOT checked in) — Phase 1/3/4 lib changes were additive and broke nothing.
39+
- First real `/codex-build` dogfood (slugify task, scratch repo under $TMPDIR, `--verifier codex` degrade since headless claude is logged out): execute phase SUCCEEDED (slugify landed, all 4 scratch tests pass), but verify phase ERRORED → runner exit 2, `verdict_present: false`, ESCALATE. codex_total_tokens=21497 (token capture works); claude_* totals=0 (ladder not exercised on the degrade path). One re-kick (`--parent-run`, lineage recorded) hit the same error. NO ACCEPT data point yet.
40+
- Real bug found AND FIXED (2026-06-12): the documented `--verifier codex` degrade leaked the preset's `VERIFIER_MODEL=fable` into the codex seat (`apply_seat_env`, dev-review.sh:1370-1372) → codex on a ChatGPT account returned HTTP 400 "The 'fable' model is not supported". Fix = cross-agent leak guard in `apply_seat_env` + `resolve_seat_model_string` (drop a wrong-kind model+effort pair as a unit; codex seat falls back to `codex:(default)@(default)`). Sim scenario (h) added (preset-expansion-simulation.sh now 8/8; run-all 25/25 green). Re-run proof: fable 400 GONE, execute SUCCEEDED, scratch run-tests ALL PASS, codex_total_tokens=17237, wall 54s — BUT verdict still null: the verify seat now hit a SEPARATE, pre-existing schema 400 (`invalid_json_schema`: nested `issues.items` missing `additionalProperties:false` in skills/dev-review/schemas/review-verdict.json). Seat fix proven; degrade path was blocked one layer deeper by the schema bug. Detail in .planning/research/2026-06-12-token-evidence.md.
41+
- Schema 400 FIXED (2026-06-12): OpenAI strict structured-output requires `additionalProperties:false` + a `required` list covering EVERY property on EVERY object node. `issues.items` was missing both; top-level `required` omitted `scope_creep_detected`/`iteration_notes`. Tightened all THREE canonical copies identically (schemas/, runners/codex-ps/schemas/, skills/dev-review/schemas/) so the drift guard stays green; shell `validate_review_verdict` is unaffected (stays loose, independent of this file). DEGRADE-PATH E2E NOW GREEN: real /codex-build (subtract-helper task, scratch repo under $TMPDIR, `--verifier codex`, --branch auto) → exit 0, verify OK, **verdict APPROVED conf 96**, verdict.json complete (all 6 strict fields), tokens execute=30606 verify=14485 codex_total=45091, wall 59s, scratch run-tests ALL 4 PASS. First full ACCEPT-path evidence row (degrade path). run-all 25/25 green. First ACCEPT data point logged in .planning/research/2026-06-12-token-evidence.md.
42+
- Remaining Phase 6 (HUMAN + follow-up): `claude /login` on this Mac (unblocks full ladder + non-zero claude_* tokens; degrade-path claude_* are 0 by design); REVISE→ACCEPT row still owed (ESCALATE + ACCEPT now evidenced); an interactive `/dev-review` baseline for the 50%-claim denominator.
43+
Last activity: 2026-06-12 -- Phase 6: schema 400 fixed (3 copies); degrade-path E2E green, first ACCEPT (.planning/research/2026-06-12-token-evidence.md)
44+
Working directories: `~/Project/co-evolution/` on the Mac (per-machine clone; SMB checkout `/Volumes/Project/co-evolution` is sync-only), `C:/Users/alan/Project/co-evolution-*` on the PC
3445

3546
macOS baseline before v1.3 fixes: scorer-verification 11/14; code-proposer sim 1/16; pr-emitter sim 4/12; template-proposer sim 1/8; revise-loop sim aborts. Root causes: bash 3.2 (mapfile, source <(…)), BSD sed GNU-isms. Target after Phase 0.5: all green on macOS.
3647

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# R1: claude -p JSON Envelope — Field Reference
2+
3+
**Date:** 2026-06-12
4+
**Phase:** v1.5 Phase 0
5+
**Purpose:** Pin the exact JSON output structure of `claude -p --output-format json` for Phase 4 token-capture parsing.
6+
7+
## Command run
8+
9+
```
10+
claude -p --output-format json --model claude-haiku-4-5-20251001 "Reply with exactly: PING"
11+
```
12+
13+
## Verbatim output (not-logged-in state; all usage fields present and zero)
14+
15+
```json
16+
{"type":"result","subtype":"success","is_error":true,"api_error_status":null,"duration_ms":1103,"duration_api_ms":0,"num_turns":1,"result":"Not logged in · Please run /login","stop_reason":"stop_sequence","session_id":"4642f382-c299-4d72-bcaf-3e7bca396c7d","total_cost_usd":0,"usage":{"input_tokens":0,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"output_tokens":0,"server_tool_use":{"web_search_requests":0,"web_fetch_requests":0},"service_tier":"standard","cache_creation":{"ephemeral_1h_input_tokens":0,"ephemeral_5m_input_tokens":0},"inference_geo":"","iterations":[],"speed":"standard"},"modelUsage":{},"permission_denials":[],"terminal_reason":"completed","fast_mode_state":"off","uuid":"def45115-84e3-4e92-a601-20e0dae05dda"}
17+
```
18+
19+
Exit code: 1 (error because not logged in; envelope still emitted on stdout)
20+
21+
## Top-level fields
22+
23+
| Field | Type | Notes |
24+
|---|---|---|
25+
| `type` | string | Always `"result"` |
26+
| `subtype` | string | `"success"` even on error |
27+
| `is_error` | bool | `true` when auth fails or API error |
28+
| `api_error_status` | null/int | HTTP status code on API errors; null here |
29+
| `duration_ms` | int | Wall time in ms |
30+
| `duration_api_ms` | int | API time in ms |
31+
| `num_turns` | int | Number of conversation turns |
32+
| `result` | string | The model's text output (or error message) |
33+
| `stop_reason` | string | e.g. `"stop_sequence"`, `"end_turn"` |
34+
| `session_id` | string | UUID |
35+
| `total_cost_usd` | float | Total cost in USD (0 when not logged in) |
36+
| `usage` | object | Token usage breakdown — see below |
37+
| `modelUsage` | object | Per-model usage breakdown (empty when not logged in) |
38+
| `permission_denials` | array | Tool permission denial events |
39+
| `terminal_reason` | string | `"completed"` |
40+
| `fast_mode_state` | string | `"off"` |
41+
| `uuid` | string | Run UUID |
42+
43+
## usage subfields
44+
45+
| Field | Type | Notes |
46+
|---|---|---|
47+
| `input_tokens` | int | Prompt input tokens |
48+
| `cache_creation_input_tokens` | int | Cache write tokens |
49+
| `cache_read_input_tokens` | int | Cache hit tokens |
50+
| `output_tokens` | int | Response tokens |
51+
| `server_tool_use.web_search_requests` | int | Web search count |
52+
| `server_tool_use.web_fetch_requests` | int | Web fetch count |
53+
| `service_tier` | string | `"standard"` or `"priority"` |
54+
| `cache_creation.ephemeral_1h_input_tokens` | int | 1-hour ephemeral cache write tokens |
55+
| `cache_creation.ephemeral_5m_input_tokens` | int | 5-min ephemeral cache write tokens |
56+
| `inference_geo` | string | Inference geography code |
57+
| `iterations` | array | Per-iteration usage (for multi-turn) |
58+
| `speed` | string | `"standard"` |
59+
60+
## Notes for Phase 4
61+
62+
- All token fields live under `.usage`. Phase 4's `invoke_claude` gated JSON mode should extract: `.usage.input_tokens`, `.usage.output_tokens`, `.usage.cache_creation_input_tokens`, `.usage.cache_read_input_tokens`.
63+
- `.total_cost_usd` is a direct top-level field, not nested.
64+
- The envelope is always emitted to **stdout** even when `is_error=true`.
65+
- The exit code is 1 on auth error; Phase 4 must handle non-zero exit with usable envelope (capture stdout regardless of exit code using `|| true`).
66+
- **Limitation:** This capture is from a not-logged-in shell. When logged in, `modelUsage` will be populated and `iterations` may have per-turn breakdown. Field names are stable across auth state.
67+
68+
## Auth status at capture time
69+
70+
`claude whoami` returns: `Not logged in · Please run /login`
71+
(The Mac's interactive Claude Code sessions authenticate through the Electron app, not this shell. The sub-agent shell does not carry the session token.)

0 commit comments

Comments
 (0)