P0: auth-gate head-scan + factual doc corrections - #39
Conversation
validate_agent_artifact only failed on an auth banner in the output when the whole file was under 50 words (deb4669's anti-false-positive ceiling), so a real login/auth error page longer than 50 words fell through and was accepted as the document. Replace the whole-file length heuristic with a strict, line-anchored head-scan (output_contains_auth_banner) over the first ~20 non-empty lines: a banner that leads the output is fatal at any length, while a long legitimate document echoing auth phrases mid-body still passes. Markdown-quoted banners in the head are skipped (``` fences, 4-space/tab indents, > blockquotes) so an auth-handling plan that quotes the banner as an example is not misread — real banners print at column 0 outside any code context; bold **...** lead-ins still match. The broad matcher still guards the stderr/empty-output paths. Adds tests/auth-gate-simulation.sh pinning both dimensions: A1/A3/A5 fail against the old code, A2/A4 guard over-eager matching, A6-A8 pin the quoted-example skips, A9 pins fence-toggle correctness. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bring the docs in line with current truth so agents and readers stop acting on stale facts: the marker convention said markers "auto-expire" (they do not — bounce stops at MAX_BOUNCES=2 and unresolved markers are reported); AGENTS.md called agent-bouncer the "primary executable" and claimed the Claude adapter is hard-coded to claude-opus-4-6 (it is the default, overridable via CLAUDE_MODEL / --claude-model); README/CLAUDE still framed agent-bouncer as a first-class runner. Demote it to a legacy runner, add a CLAUDE.md Status block (v1.4 npm/MCP + v1.5 Phase 6 pending; default model), note in README that 3-OS CI covers stubbed sims not live LLMs, and mark the pre-v1.0 codebase snapshots (no-CI/no-tests) historical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fbda24b60c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| [[ -n "$eligible" ]] || return 1 | ||
|
|
||
| grep -qiE \ | ||
| '^[[:space:]>*•[:punct:]]*(Not logged in|You are not logged in|Please run [^[:space:]]*login|Please sign in|Please log ?in|Login required|Authentication failed|Failed to authenticate|authentication_error|Your organization does not have access|Invalid API key|Session (has )?expired)' \ |
There was a problem hiding this comment.
When an auth failure is written to stdout in a form that file_contains_auth_failure() still recognizes, such as Not authenticated. Please run claude login or Please run claude login, this new output-path regex does not match: Not authenticated is no longer an alternative, and Please run [^[:space:]]*login only allows a single no-space token like /login. In those cases validate_agent_artifact now returns 0 for a short auth banner and the bounce path can copy it into the document, which is the failure this gate is meant to prevent.
Useful? React with 👍 / 👎.
| [[ -n "$eligible" ]] || return 1 | ||
|
|
||
| grep -qiE \ | ||
| '^[[:space:]>*•[:punct:]]*(Not logged in|You are not logged in|Please run [^[:space:]]*login|Please sign in|Please log ?in|Login required|Authentication failed|Failed to authenticate|authentication_error|Your organization does not have access|Invalid API key|Session (has )?expired)' \ |
There was a problem hiding this comment.
Because the prefix class includes all [:punct:], valid Markdown headings or list items in the first 20 non-empty lines are reduced to the auth phrase and rejected, e.g. an agent-produced document starting with # Authentication failed handling plan makes validate_agent_artifact return 2 even though it is a normal artifact about auth behavior. This blocks exactly the auth-handling documentation/plans the new tests try to allow; limit the accepted decoration to real banner wrappers instead of every punctuation character.
Useful? React with 👍 / 👎.
1 participant
Phase 0 of the workflow-upgrade plan (plan @ e6db065,
.planning/notes/2026-07-05-workflow-upgrade-plan.md).What changed
lib/co-evolution.sh:579-628— newoutput_contains_auth_banner(): strict anchored auth-banner detector over the first 20 non-empty lines of agent output, skipping fenced/indented/blockquoted (quoted-example) lines; replaces the <50-word whole-file ceiling invalidate_agent_artifact(closes the >50-word auth-page blind spot). Broad matcher retained for stderr/empty paths; rc semantics (0/1/2) unchanged at all 8 call sites.tests/auth-gate-simulation.sh— new hermetic sim, 9 scenarios: long auth page rejected (A1), legit docs mentioning/quoting auth banners accepted (A2, A4, A6-A8), authentication_error page rejected (A3), E2E bouncer abort (A5), fence-toggle guard (A9).CLAUDE.md(marker-expiry claim → actual MAX_BOUNCES behavior; agent-bouncer demoted to legacy; 3-line Status block),AGENTS.md:59,71,133,README.md:118-121,204-205(CI = hermetic sims caveat), historical-snapshot headers on.planning/codebase/CONCERNS.md/STACK.md.Verification
Accepted risk (documented, strictly no worse than master): an auth banner arriving after 20+ non-empty preamble lines, or prefixed by raw ANSI escapes, falls back to master's accept behavior.
🤖 Generated with Claude Code