Skip to content

Release pipeline: failures are silent, and NPM_TOKEN is an unused live credential #562

Description

@emal-avala

Two hardening items from the v0.29.0 release, where three of the five publish targets failed and one had been failing silently for three weeks.

1. Release failures are invisible

The Homebrew tap job failed on eight consecutive releases (v0.25.0 through v0.29.0) and nobody noticed. The tap formula sat at 0.25.3 from 2026-07-07 until it was fixed by hand on 2026-07-28 — four versions stale.

It hid because the other jobs go green, the run summary is a single red mark on a workflow nobody opens after a tag push, and brew users get an old version rather than an error. Meanwhile the release itself looks successful: binaries, GitHub release and Docker all published.

The v0.29.0 run had two failures — crates.io (revoked token) and Homebrew (permissions) — and both were only found because someone went looking.

Suggested fix: notify on release-workflow failure. Anything that reaches a human beats a red square in a tab: a Slack post to #eng, an issue opened automatically, or a scheduled check that compares the tap formula version and the published crates.io/npm versions against the newest tag and shouts when they diverge. The last option also catches a job that silently no-ops.

2. NPM_TOKEN is a live credential that nothing uses

gh secret list shows NPM_TOKEN, last updated 2026-04-05. git grep NPM_TOKEN over .github/ and npm/ returns exactly one hit — a comment in release.yml:197 explaining its absence. npm has published via OIDC trusted publishing since v0.23.0 (permissions: id-token: write, npm publish --provenance, and an explicit "No NODE_AUTH_TOKEN" comment).

So it is an npm credential with publish rights to @avala-ai/agent-code, sitting in CI, that no code path can vouch for. It should be deleted.

For reference, after #555 the release pipeline's credential surface is:

Target Auth
crates.io OIDC trusted publishing (both crates)
npm OIDC trusted publishing
Docker GITHUB_TOKEN
GitHub release GITHUB_TOKEN
Homebrew tap HOMEBREW_TAP_TOKEN — the only long-lived secret left

Note for whoever picks up the Homebrew half

The eight-release failure was misdiagnosed twice as a token problem — first "expired", then "wrong scope". Neither was true. The token's owner (avala-dev) had read on avala-ai/homebrew-tap while the org's default_repository_permission is none. A correct scope on an account without write produces an identical 403 Resource not accessible by personal access token.

The two-second check that would have found it immediately:

curl -s -H "Authorization: token $TOK" \
  https://api.github.com/repos/avala-ai/homebrew-tap | jq .permissions

push: false says everything. This is now recorded in RELEASING.md (#555), but it is worth a /doctor-style preflight in the release workflow itself: verify write access before the tag drives five publish jobs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions