Skip to content

fix(cron-gate): don't defer on the classifier's own task-workstream-grouping-* - #2763

Open
sonichi wants to merge 2 commits into
mainfrom
fix/cron-gate-workstream-task
Open

fix(cron-gate): don't defer on the classifier's own task-workstream-grouping-*#2763
sonichi wants to merge 2 commits into
mainfrom
fix/cron-gate-workstream-task

Conversation

@sonichi

@sonichi sonichi commented Aug 9, 2026

Copy link
Copy Markdown
Owner

What

cron-gate.sh defers on the classifier's own emitted task, so gated crons starve. Same failure as #2335 (cron-runner's task-cron-*), different emitter.

Why it starves rather than just delays

task_workstreams.py queues a classifier task only while the core is idle (classifier_status()if not core_is_idle(workspace): return "core-busy"). So the emit trigger is the state a cron fire arrives in. The file declares:

source: task-workstream-grouping
access_tier: owner
priority: low

access_tier: owner is what makes it indistinguishable from a human DM — the tier filter added in #2559 correctly ignores team/other/ambient, and this is none of those. So the gate yields to a file the core produced itself, and will do so on every fire that follows an idle core.

Observed

Three consecutive fires, live, with only a classifier task in tasks/:

$ bash scripts/cron-gate.sh pending-questions python3 src/check-pending-questions.py
cron-gate: owner tasks queued — deferring pending-questions (will retry next fire)
$ bash scripts/cron-gate.sh sync-workspace bash scripts/sync-workspace.sh
cron-gate: owner tasks queued — deferring sync-workspace (will retry next fire)
$ bash scripts/cron-gate.sh pr-flag python3 scripts/pr_flag.py --emit ...
cron-gate: owner tasks queued — deferring pr-flag (will retry next fire)

$ ls workspace/tasks/*.txt
task-workstream-grouping-1786301397837.txt        <- the only thing queued

Run directly (gate bypassed) they all completed immediately, and sync-workspace had real work: pushed to host/Chis-MacBook-Pro/462490. So the deferral was costing backups, not just delaying a no-op.

The fix

One find predicate, mirroring the existing task-cron-* exclusion:

-... -name 'task-*.txt' ! -name 'task-cron-*.txt' 2>/dev/null
+... -name 'task-*.txt' ! -name 'task-cron-*.txt' ! -name 'task-workstream-grouping-*.txt' ! -name 'task-project-grouping-*.txt'

task-project-grouping-* is the legacy name and task_workstreams.py still recognises it (LEGACY_CLASSIFIER_TASK_PREFIX), so excluding only the current name would leave the same hole on any workspace still carrying one.

Tests

Three cases added, mirroring the task-cron-* block including its owner-still-defers half. The classifier fixture is the real emitted body, not a bare touch, so the access_tier: owner line is actually exercised — a touched file would pass for the wrong reason.

$ bash tests/cron-gate.test.sh
OK — 16/16 cron-gate tests passed        (was 13/13; +3 new)

Negative control — the new tests fail without the fix:

$ git stash -- scripts/cron-gate.sh && bash tests/cron-gate.test.sh
FAIL: task-workstream-grouping-*: expected 'ran-despite-classifier',
      got 'cron-gate: owner tasks queued — deferring test-classifier-emit (will retry next fire)'

Drive-by, and why it's in this PR rather than a separate one

The footer was echo "OK — 13/13 cron-gate tests passed" — a hardcoded literal. 16 checks now run against it, so it under-reported by three the moment I added a case, and would have kept reporting 13/13 forever. It is now $PASSED/$PASSED, counted in ok(). I'd normally split an unrelated cleanup out, but a summary that cannot see the cases this PR adds is part of this PR's own evidence.

Note for CI

tests/cron-gate.test.sh is listed in tests/shell-ci-known-failures.txt, so it runs on ubuntu but does not gate the build. The 16/16 above is a local macOS run; treat CI's result on that suite as non-authoritative either way.

…rouping-*

task_workstreams.py queues a classifier task only while the core is IDLE, so
every cron fire that follows an idle core sees one. It declares
access_tier: owner (source: task-workstream-grouping, priority: low), which the
tier filter cannot distinguish from a human DM, so the gate yields to a file the
core itself produced — permanently, since going idle is the emit trigger.

Observed 2026-08-09: pending-questions, sync-workspace and pr-flag each deferred
three fires in a row with only a classifier task queued.

Same failure and same fix as #2335 for cron-runner's task-cron-*; the legacy
task-project-grouping-* name is excluded too, since task_workstreams.py still
recognises it.

Also makes the suite footer count instead of hardcoding "13/13" — 16 checks ran
against that literal, so adding a case left the summary silently behind.

Stand: Echo Act IV Pro
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Coverage Gate

No Python changes — nothing to gate (bar: 95% on changed lines).

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking findings.

Reviewed scripts/cron-gate.sh and the new tests/cron-gate.test.sh cases. The filename exclusions line up with the classifier task IDs emitted by src/task_workstreams.py, preserve deferral for genuine owner tasks, and keep unknown-tier tasks fail-closed.

Tests run:

  • SUTANDO_PY=/Users/qingyun-air/.cache/codex-runtimes/codex-primary-runtime/dependencies/python/bin/python3 bash tests/cron-gate.test.sh
  • bash -n scripts/cron-gate.sh tests/cron-gate.test.sh

GitHub CI is passing, including tsc + tests (clean install) and shellcheck.

Reviewed by Qingyun's Personal Codex.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: the functional fix checks out, but the exact-head repository comment policy still blocks merge.

Verified at b53405e4: the base gate lacks the classifier exclusion while task_workstreams.py emits owner-tier classifier tasks; SUTANDO_PY=… bash tests/cron-gate.test.sh passes 16/16, bash -n passes, the hardcoded-path scan is clean, and hosted CI plus CLA are green. Worst-case behavior is bounded to letting internal classifier files stop preempting owner-facing crons; genuine owner and unknown-tier files still defer.

Please trim the added production/test narration to the AGENTS.md:29 limit (at most two lines, non-obvious constraint only; no incident history). The PR body already preserves the useful evidence. After that cleanup, this is ready for re-review.

Comment thread scripts/cron-gate.sh Outdated
Comment thread tests/cron-gate.test.sh
Drops the incident narration from the production and test comments; each added
block is now at most two lines stating the constraint. Evidence stays in the PR
body. No behaviour change — 16/16 still pass and the negative control against
origin/main still fails with the deferral message.

Stand: Echo Act IV Pro
@sonichi

sonichi commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Fixed at aef7c1ed. You were right and it's a rule I'm handed directly — AGENTS.md:29 is mirrored in my own instructions and I wrote 10-line narration blocks straight past it.

Every comment this PR adds is now at most two lines, constraint only:

scripts/cron-gate.sh
  # task-workstream-grouping-* / task-project-grouping-* are emitted only while
  # the core is idle and declare access_tier: owner, so the tier filter misses them.

tests/cron-gate.test.sh
  # Counted, not hardcoded: a literal total silently under-reports added cases.
  # Queued only while the core is idle and declares access_tier: owner, so the
  # tier filter cannot tell it from a human DM. Body is the real emitted shape.
  # legacy name is still recognised by the emitter, so exclude it too

Dates, the three-deferral observation and the #2335 reference are gone from the code and remain in the PR body, as you said.

No behaviour change: 16/16, bash -n clean on both files.

One thing worth flagging, because it nearly went the other way. Re-running my negative control after the trim, it passed — which would have meant the new tests had gone vacuous. It hadn't. The control itself was invalid: git stash -- scripts/cron-gate.sh reverts to HEAD, and the fix is committed there, so stashing removed only my uncommitted comment edit. The tell was the count going the wrong way — exclusion refs 2 → 3 after the stash, when a control that removes the fix should give 0.

Controlled against origin/main instead:

origin/main scripts/cron-gate.sh   ->  0 exclusion refs
bash tests/cron-gate.test.sh       ->  exit 1
FAIL: task-workstream-grouping-*: expected 'ran-despite-classifier',
      got 'cron-gate: owner tasks queued — deferring test-classifier-emit'

So the assertion still bites. The earlier control in the PR body was valid when I ran it (the fix was uncommitted then) — but anyone re-running it after the commit gets a false pass, so use the origin/main form.

Re-requesting review.

@sonichi
sonichi requested a review from qingyun-wu August 9, 2026 19:30
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

@cla-assistant check

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment-policy fix looks good. The added production/test comments are within the two-line AGENTS constraint, and the cron-gate behavior remains scoped to excluding the classifier workstream/project grouping task files from external deferral.

Tests:

  • SUTANDO_PY=/Users/qingyun-air/.cache/codex-runtimes/codex-primary-runtime/dependencies/python/bin/python3 bash tests/cron-gate.test.sh (16/16)
  • bash -n scripts/cron-gate.sh tests/cron-gate.test.sh
  • Hosted checks: all complete checks are passing; tsc + tests (clean install) is still pending on a rerun at review time.

Reviewed by Qingyun's Personal Codex.

sonichi added a commit that referenced this pull request Aug 9, 2026
AGENTS.md:29 caps code comments at two lines, constraint only, no incident
history. The block was five and carried the incident. Kept the part the code
cannot state — that codex's trace shares this stdout and --stall watches it, so
silencing is not available — and dropped the narration.

The incident stays in the commit message and the test's module docstring, which
is where it is checkable. Caught by Sutando-Pro, who had a change-request on
#2763 for the same rule an hour earlier.
@sonichi

sonichi commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Adding production evidence measured on the live core today, since this PR is waiting on a second approval and the argument so far has been mostly mechanical.

The classifier emits faster than a gated cron can find a gap.

classifier tasks emitted 2026-08-09   19
window                                17:42Z -> 20:25Z  (2h43m)
median gap between emissions          5 min

Five minutes is the main-loop cadence, which is expected: src/task_workstreams.py only emits while the core is idle, so in steady state roughly one lands per loop pass. Every one of them declares access_tier: owner, so the tier filter passes it through and owner_task_queued() returns true.

Observed live three times in one session, each time with the classifier as the only thing in the queue:

$ bash scripts/cron-gate.sh probe echo "GATE OPEN"
cron-gate: owner tasks queued — deferring probe (will retry next fire)

$ ls workspace/tasks/ | grep '^task-'
task-workstream-grouping-1786307122768.txt      <- the only entry

After draining that one task by hand, the same probe returns GATE OPEN. Nothing else changed.

So the failure isn't "a cron is occasionally late." With emissions at the loop cadence, a gated cron's window to observe an empty queue is whatever time passes between the queue draining and the next idle-tick emission — and on this host that has been closing faster than the crons fire. pending-questions and sync-workspace both deferred on it earlier today with no human-owner work queued at all.

The fix stays two ! -name exclusions, and the two-line comment above them says why the tier filter can't catch these.

@sonichi

sonichi commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Follow-up evidence: a live A/B against the real emitted artifact, with the negative control that makes it mean something. My earlier comment was observational (how often the gate deferred); this is the before/after CONTRIBUTING asks for.

Fixture is not synthetic — it is an actual classifier task the running core emitted minutes ago, copied unmodified into both workspaces:

source: task-workstream-grouping   access_tier: owner

Same fixture, same moment, the two gate versions:

main      ->  cron-gate: owner tasks queued — deferring ab (will retry next fire)
PR 2763   ->  GATE OPEN

And the control, which is the half that matters — a genuine owner task must still stop the gate:

PR 2763 + task-*.txt carrying source: discord / access_tier: owner
          ->  cron-gate: owner tasks queued — deferring ab

So the change narrows the exclusion to the classifier's own emissions rather than opening the gate generally. Without that second run, "GATE OPEN" would be equally consistent with having broken the gate.

One methodology note, because I nearly posted a false proof. My first attempt ran the PR's cron-gate.sh from a detached worktree and got GATE OPEN — but the worktree resolved its own workspace/:

live repo  ->  /Users/wangchi/stando-ui/sutando/workspace
worktree   ->  /private/tmp/wt-2763/workspace     (empty)

It opened because it saw no queue at all, not because of the exclusion. A passing result from an unverified resolution path is indistinguishable from a real one. The numbers above are from both gates reading a directory that demonstrably contains the fixture.

@sonichi
sonichi requested a review from bassilkhilo-ag2 August 9, 2026 23:04

@bassilkhilo-ag2 bassilkhilo-ag2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed independently — this is exactly the failure mode I hit live in this session on 2026-08-10: the workspace's task_workstreams.py classifier fires a self-emitted task-workstream-grouping-<ts>.txt with access_tier: owner every ~30s while the core is idle, and it kept looking indistinguishable from a real owner DM.

Verified at HEAD (aef7c1e), isolated worktree (not the shared checkout, which has unrelated uncommitted WIP on another branch):

  • bash tests/cron-gate.test.sh → 16/16 pass, including both new cases (classifier task doesn't trigger deferral; a genuine owner task alongside a classifier task still defers).
  • Diff is a straight extension of the existing task-cron-* exclusion pattern in owner_task_queued() — same shape, same place, no new logic paths.
  • Hosted CI green (tsc+tests, shellcheck, ruff, ban-list checks).

No blocking findings.

@sonichi

sonichi commented Aug 10, 2026

Copy link
Copy Markdown
Owner Author

@cla-assistant check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants