Skip to content

feat(ci): test the Codex support window at both ends, and report upstream drift (#408) - #550

Merged
SUaDtL merged 1 commit into
mainfrom
feat/408-codex-host-version-window
Jul 30, 2026
Merged

feat(ci): test the Codex support window at both ends, and report upstream drift (#408)#550
SUaDtL merged 1 commit into
mainfrom
feat/408-codex-host-version-window

Conversation

@SUaDtL

@SUaDtL SUaDtL commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Three of #408's four acceptance criteria, all credential-free. AC-2 is not here, and says so.

AC-1 — the supported range had an unexercised edge

The manifest advertised Codex >= 0.143.0; live-verified on 0.144.1 while CI installed exactly one version. The lower bound of the supported range rested on a single manual check from July. A range nothing exercises at its edge is a claim, not a guarantee.

The required real-host lane now runs at both ends — 0.143.0 and 0.145.0. Both were verified installable from npm before matrixing on them.

Taken via AC-1's first arm rather than its second: narrowing the claim to one tested version would have told working 0.143 installs they are unsupported.

The contract no longer guards the sentence

It used to assert the literal string "0.144.1" — which made the manifest's support claim a statement about a past afternoon. It now derives the requirement from the workflow: the description must name every version the required lane installs. Bumping the matrix without amending the claim fails, and so does the reverse.

AC-3 — a check that agreed with itself by construction

The real-host check verified that the installed hook manifest matched the source manifest. But the install is a fresh copy of that same source, so for anything source declares the two agree by construction.

Measured against a real host — I added a hook to source with no script behind it:

mutant (source claims a hook the install lacks): SURVIVED (exit=0)

It can catch the install dropping a file, which is its stated purpose, and nothing else.

So every plugin-root-relative script the installed manifest points at must now exist in the install. A hook the host cannot execute is indistinguishable from one that allowed the operation — precisely the failure this lane exists to make visible.

Verified against codex-cli 0.145.0 on this machine:

9/9 checks pass on a clean install

mutant (declared hook with no script): KILLED
::error::CODEX-HOST-HOOK-SCRIPTS: 5 script(s) referenced,
         missing from the install: ['hooks/does-not-exist.py']

AC-4 — the pinned lane cannot report drift

By construction: it pins. An advisory lane now installs @openai/codex@latest and runs the identical check.

Awaited by ci-passed so the run settles, never in required_results — an upstream release must not be able to block a merge. It absorbs failure at step level rather than carrying a job-level continue-on-error, which would still render the check run as FAILURE (#381's lesson, and the Pi canary's shape).

Already earning it: npm latest is 0.146.0 while the required lane pins 0.145.0 and the manifest said live-verified on 0.144.1. Nothing in CI reported that.

AC-2 stays open, and needs a decision rather than more work

Proving a hook fires needs a turn, a turn needs a model, and a provider credential cannot gate fork PRs. That boundary was already documented in this lane when the credential-free half shipped; this change does not move it.

Verification

test_codex_adapter 82, test_ci_impact 54, test_public_codex_docs 3, test_validate_codex_plugin 6, test_license_consistency 23 — all pass. Real-host check exercised locally against a live Codex, both clean and mutated.

ca-codex 0.3.1 → 0.3.2 — the manifest description is shipped payload.

…ream drift (#408)

Three of #408's four acceptance criteria, all credential-free. AC-2 is not here
and says so.

AC-1 - the manifest advertised `Codex >= 0.143.0; live-verified on 0.144.1` while
CI installed exactly ONE version. The lower bound of the supported range rested
on a single manual check from July; a range nothing exercises at its edge is a
claim, not a guarantee. The required real-host lane now runs at both ends of the
window, 0.143.0 and 0.145.0.

Taken via AC-1's FIRST arm rather than its second: narrowing the claim to one
tested version would have told working 0.143 installs they are unsupported. Both
npm versions were verified installable before matrixing on them.

The contract test no longer pins a literal. It DERIVES the requirement from the
workflow: the manifest description must name every version the required lane
installs, so bumping the matrix without amending the claim fails, and so does the
reverse. It used to assert the string "0.144.1" - guarding the sentence rather
than the fact.

AC-3 - the real-host check verified that the installed hook manifest matched the
SOURCE manifest. The install is a fresh copy of that same source, so for anything
source declares the two agree BY CONSTRUCTION. Measured against a real host: a
hook added to source with no script behind it left that check passing. It can
catch the install dropping a file, which is its stated purpose, and nothing else.

So every plugin-root-relative script the INSTALLED manifest points at must now
exist in the install. A hook the host cannot execute is indistinguishable from
one that allowed the operation, which is exactly the failure this lane exists to
make visible. Verified against codex-cli 0.145.0 on this machine: 9/9 checks pass
clean, and a declared-but-absent hook is KILLED with the missing path named.

AC-4 - the required lane PINS its versions, so by construction it can never
report that upstream moved. An advisory lane now installs @latest and runs the
identical check. Awaited by ci-passed so the run settles, never in
required_results: an upstream release must not be able to block a merge. It
absorbs failure at STEP level rather than carrying a job-level
continue-on-error, which would still render the check run as failure (#381's
lesson).

Already earning it: npm latest is 0.146.0 while the required lane pins 0.145.0
and the manifest said live-verified on 0.144.1. Nothing in CI reported that.

AC-2 remains open and needs a decision, not more work: proving a hook FIRES needs
a turn, a turn needs a model, and a provider credential cannot gate fork PRs.
That boundary was already documented in this lane; this change does not move it.

ca-codex 0.3.1 -> 0.3.2: the manifest description is shipped payload.

Claude-Session: https://claude.ai/code/session_01WJgVfZw7J81PB7mwpHyUxx
@SUaDtL
SUaDtL merged commit 11c28e8 into main Jul 30, 2026
48 checks passed
@SUaDtL
SUaDtL deleted the feat/408-codex-host-version-window branch July 30, 2026 02:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant