UX: warn when action_shas is present and --resolve is off
Problem
nboot audit runs offline by default — --resolve is off, so action
SHAs are left as placeholder values in the rendered plan. This is
intentional and documented (audit.md "By default audit runs
offline …"; the CLI --help says the same), so this isn't a hidden
bug.
The improvement opportunity: when a pack declares action_shas AND
the user runs offline, compute_diffs will report every affected
workflow as drifted because the rendered placeholders don't match
the on-disk real SHAs. The user has no on-screen cue that the
finding count would collapse if they passed --resolve. They then
either (a) treat the diff as a real drift list and waste triage
effort, or (b) eventually re-read the docs and re-run.
This is a UX gap, not a defect — the docs already cover it, but the
runtime experience doesn't reinforce it at the moment of confusion.
Proposed resolution
Single approach: smart warning at audit time.
run_audit() already has both skip_resolve and the loaded manifest
in scope at audit.py:133-136. When manifest.get("action_shas") is
non-empty and skip_resolve=True, emit a one-line note to stderr
before running compute_diffs:
Notice: --resolve is off and pack '<name>' declares N action SHA(s).
Workflow files rendered with placeholder SHAs may show as drift even
when the target is fully conforming. Re-run with --resolve (requires
gh) to compare against real SHAs.
Don't fail; don't suppress findings; let the user decide.
Pair with a one-paragraph addition to audit.md "Operational notes"
linking the flag and the manifest field.
Why "warning, not docs-only"
The docs path was suggested but rejected — the existing docs already
state the offline default clearly. Repeating the warning in CLI help
won't reach users who don't pass --help. A runtime warning fires
exactly when the user is about to be confused.
A. Smart warning (preferred)
When the rendered plan contains action-SHA placeholders AND
--resolve is off, emit a one-line warning to stderr:
Notice: --resolve off; N action-SHA placeholders in the rendered
plan will report as drift if the target was rendered online.
Re-run with --resolve to compare against real SHAs (requires gh).
Don't fail; let the user decide. Pair with a doc paragraph in
docs/reference/audit.md's "Operational notes" section.
B. Documentation only
Keep current behaviour, add a clear caveat in the --help text and
docs explaining that audits against pack-rendered repos generally
need --resolve to be meaningful.
Acceptance criteria
Effort
~30 minutes including tests + doc paragraph.
Source
Bot review on PR #51 (Grippy audit.py:175 MEDIUM
"Network-Skipping Mode May Cause Audits to Miss SHA Drift"),
sharpened post-Codex review (the original "silent footgun" framing
was wrong — docs do cover this — so this is a UX-warning
improvement, not a bug fix).
UX: warn when
action_shasis present and--resolveis offProblem
nboot auditruns offline by default —--resolveis off, so actionSHAs are left as placeholder values in the rendered plan. This is
intentional and documented (
audit.md"By defaultauditrunsoffline …"; the CLI
--helpsays the same), so this isn't a hiddenbug.
The improvement opportunity: when a pack declares
action_shasANDthe user runs offline,
compute_diffswill report every affectedworkflow as drifted because the rendered placeholders don't match
the on-disk real SHAs. The user has no on-screen cue that the
finding count would collapse if they passed
--resolve. They theneither (a) treat the diff as a real drift list and waste triage
effort, or (b) eventually re-read the docs and re-run.
This is a UX gap, not a defect — the docs already cover it, but the
runtime experience doesn't reinforce it at the moment of confusion.
Proposed resolution
Single approach: smart warning at audit time.
run_audit()already has bothskip_resolveand the loaded manifestin scope at
audit.py:133-136. Whenmanifest.get("action_shas")isnon-empty and
skip_resolve=True, emit a one-line note to stderrbefore running
compute_diffs:Don't fail; don't suppress findings; let the user decide.
Pair with a one-paragraph addition to
audit.md"Operational notes"linking the flag and the manifest field.
Why "warning, not docs-only"
The docs path was suggested but rejected — the existing docs already
state the offline default clearly. Repeating the warning in CLI help
won't reach users who don't pass
--help. A runtime warning firesexactly when the user is about to be confused.
A. Smart warning (preferred)
When the rendered plan contains action-SHA placeholders AND
--resolveis off, emit a one-line warning to stderr:Don't fail; let the user decide. Pair with a doc paragraph in
docs/reference/audit.md's "Operational notes" section.B. Documentation only
Keep current behaviour, add a clear caveat in the
--helptext anddocs explaining that audits against pack-rendered repos generally
need
--resolveto be meaningful.Acceptance criteria
manifest["action_shas"]is non-empty andskip_resolveis True,
run_auditemits a one-line note to stderr namingthe count and the recommended remediation (
--resolve).--resolveis passed (online), no notice is emitted.action_shas, no notice is emittedregardless of
--resolve.docs/reference/audit.md"Operational notes" gains aparagraph linking the manifest field and the flag, with a
copy-pasteable
--resolveexample.tests/test_audit.pyasserts the notice is emitted(capsys / caplog) only in the warning-eligible case.
Effort
~30 minutes including tests + doc paragraph.
Source
Bot review on PR #51 (Grippy
audit.py:175MEDIUM"Network-Skipping Mode May Cause Audits to Miss SHA Drift"),
sharpened post-Codex review (the original "silent footgun" framing
was wrong — docs do cover this — so this is a UX-warning
improvement, not a bug fix).