Nightly wheels #74
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Nightly wheels | |
| # Build a release-candidate wheel + sdist from the current main every night and | |
| # publish them to a single rolling "dev-wheels" pre-release, so customers can | |
| # install pre-release fixes between stable releases: | |
| # | |
| # pip install "amd-aorta==X.Y.ZrcYYYYMMDD" \ | |
| # -f https://github.com/ROCm/aorta/releases/expanded_assets/dev-wheels | |
| # | |
| # The rc version is derived at build time from the latest release tag | |
| # (scripts/bump_version.py + setuptools_scm's PRETEND_VERSION) and is NOT | |
| # committed anywhere. Old assets are pruned by cleanup_releases.yml. | |
| on: | |
| schedule: | |
| - cron: "0 11 * * *" # 11:00 UTC daily | |
| workflow_dispatch: | |
| permissions: | |
| contents: write # required to create/update the dev-wheels release + assets | |
| # Serialize every run that mutates the shared, rolling dev-wheels release/tag. | |
| # cleanup_releases.yml shares this group, so a nightly publish, a cleanup prune, | |
| # and any overlapping manual dispatch / cron run can never race on force-pushing | |
| # the `dev-wheels` tag or uploading/deleting its assets (which would leave the | |
| # tag pointing at an unpredictable commit or a mixed artifact set). Do NOT cancel | |
| # the in-progress run: a publish interrupted mid tag-push/upload is worse than a | |
| # brief queue wait, so newer runs queue behind it instead. | |
| concurrency: | |
| group: dev-wheels-release | |
| cancel-in-progress: false | |
| jobs: | |
| nightly: | |
| # Guard what can publish to the *shared* canonical dev-wheels release: | |
| # * scheduled cron: canonical repo only; | |
| # * manual dispatch on the canonical repo: only from `main` (a manual run | |
| # against an arbitrary branch must not stamp non-main code into the | |
| # shared rolling pre-release); | |
| # * manual dispatch on a fork: always allowed -- it publishes to the | |
| # fork's own dev-wheels release, so it's safe for testing. | |
| if: >- | |
| (github.event_name == 'schedule' && github.repository == 'ROCm/aorta') || | |
| (github.event_name == 'workflow_dispatch' && | |
| (github.repository != 'ROCm/aorta' || github.ref == 'refs/heads/main')) | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Check out code | |
| uses: actions/checkout@v5 | |
| with: | |
| fetch-depth: 0 # full history + tags so setuptools_scm / bump_version see the latest release | |
| - name: Set up Python | |
| uses: actions/setup-python@v6 | |
| with: | |
| python-version: "3.11" | |
| - name: Stamp release-candidate version | |
| run: | | |
| # rc of the NEXT release above the latest tag (e.g. latest v0.2.0 -> | |
| # 0.2.1rcYYYYMMDD), so the nightly sorts as a pre-release of the version | |
| # it will become -- not an rc of an already-released version. The value | |
| # is fed to setuptools_scm via PRETEND_VERSION for this build only; it is | |
| # never written to any file. | |
| suffix="rc$(date -u +%Y%m%d)" | |
| if ! rc_version="$(python scripts/bump_version.py patch --suffix "${suffix}")"; then | |
| echo "::error::Failed to stamp release-candidate version." | |
| exit 1 | |
| fi | |
| echo "SETUPTOOLS_SCM_PRETEND_VERSION_FOR_AMD_AORTA=${rc_version}" >> "$GITHUB_ENV" | |
| echo "RC_VERSION=${rc_version}" >> "$GITHUB_ENV" | |
| echo "Building nightly ${rc_version}." | |
| - name: Build wheel and sdist | |
| run: | | |
| python -m pip install --upgrade build | |
| python -m build | |
| # Fail fast if the built wheel doesn't carry the stamped rc version (e.g. a | |
| # setuptools_scm/PRETEND_VERSION misconfiguration), so a mis-versioned | |
| # artifact is never uploaded to the shared dev-wheels release. Validate the | |
| # wheel's embedded METADATA version (what pip installs) rather than a single | |
| # filename, so a future build/platform tag or extra wheel can't skew it. | |
| - name: Verify built version matches | |
| run: python scripts/check_wheel_version.py "${RC_VERSION}" | |
| # Upload the exact built dist so the nightly-eval workflow can install the | |
| # precise wheel that this run produced (rather than "latest from the index"), | |
| # keeping dashboard results attributable to this run/commit. | |
| - name: Upload built dist | |
| uses: actions/upload-artifact@v6 | |
| with: | |
| name: nightly-dist | |
| path: dist/*.whl | |
| if-no-files-found: error | |
| # softprops/action-gh-release creates the dev-wheels tag on the first run | |
| # but does NOT move it on subsequent runs, so the rolling tag (and the | |
| # release's "commit" link) would keep pointing at the commit it was first | |
| # cut from while the uploaded wheels come from the current run. Force-move | |
| # the tag to the built commit first so the tag/release metadata stays | |
| # consistent with the artifacts. Lightweight tag; the job's | |
| # `contents: write` permission (via checkout's persisted credentials) | |
| # authorizes the push. | |
| - name: Point dev-wheels tag at the built commit | |
| run: | | |
| git tag -f dev-wheels "${GITHUB_SHA}" | |
| git push --force origin "refs/tags/dev-wheels" | |
| - name: Publish to rolling dev-wheels pre-release | |
| uses: softprops/action-gh-release@v3 | |
| with: | |
| tag_name: dev-wheels | |
| name: "Nightly dev wheels" | |
| body: | | |
| Rolling pre-release wheels built nightly from `main`. These are | |
| release candidates, not stable releases. Install a specific build: | |
| ``` | |
| pip install "amd-aorta==X.Y.ZrcYYYYMMDD" \ | |
| -f https://github.com/ROCm/aorta/releases/expanded_assets/dev-wheels | |
| ``` | |
| files: | | |
| dist/*.whl | |
| dist/*.tar.gz | |
| prerelease: true | |
| make_latest: "false" # never overtake the stable "Latest" release | |
| # Rolling `aorta chat` retrieval index built from main, so a developer or CI | |
| # install (which is every internal install -- setuptools_scm makes them all | |
| # .devN versions) can `aorta chat index fetch` instead of spending ~20 minutes | |
| # embedding the tree themselves. `aorta chat index fetch` resolves a .devN | |
| # version to this asset; a released wheel resolves to its own release's asset | |
| # from release.yml instead. See "Release vs nightly" in the merge plan. | |
| # | |
| # Separate job, not a step in `nightly`: it must not be able to delay or fail | |
| # the wheel publish, and it is skipped entirely on a night when the corpus | |
| # digest is unchanged. | |
| chat-index: | |
| # Ordered after the wheel job because both write to the rolling `dev-wheels` | |
| # release and that job force-moves the tag. Running them in parallel would | |
| # race a tag move against an asset upload. A failed wheel build therefore | |
| # publishes no index either, which is the right way round. | |
| needs: nightly | |
| # HARD GUARD, part 1 of 2. The index stores every chunk's source text | |
| # verbatim, so the published asset IS a redistribution of the tree it was | |
| # built from. That is fine for MIT-licensed ROCm/aorta and an IP incident for | |
| # anything else, so the job cannot run anywhere else -- and a fork cannot | |
| # reach the canonical release either way. Part 2 is `--public-only` below, | |
| # which re-checks the remote and restricts the corpus to git-tracked files. | |
| if: >- | |
| github.repository == 'ROCm/aorta' && | |
| ((github.event_name == 'schedule') || | |
| (github.event_name == 'workflow_dispatch' && github.ref == 'refs/heads/main')) | |
| runs-on: ubuntu-latest | |
| steps: | |
| - name: Check out code | |
| uses: actions/checkout@v5 | |
| with: | |
| fetch-depth: 0 # setuptools_scm needs tags; the manifest records HEAD | |
| - name: Set up Python | |
| # 3.11 is chat's floor (Decision 13a) and the version the index is | |
| # published from, so the build matches what most users query with. | |
| uses: actions/setup-python@v6 | |
| with: | |
| python-version: "3.11" | |
| - name: Install aorta with the chat CLI extra | |
| run: python -m pip install -e ".[chat-cli]" | |
| # No torch, no nvidia-*, no chromadb. Asserted rather than trusted: the | |
| # whole point of Decision 19a is that this stack cannot pull a CUDA wheel, | |
| # and a transitive dependency change is exactly how that would come back. | |
| - name: Verify the embedding stack is torch-free | |
| run: | | |
| if python -m pip list --format=freeze | grep -Ei '^(torch|nvidia-|chromadb|sentence-transformers)'; then | |
| echo "::error::The chat-cli extra resolved a torch/CUDA/chromadb dependency. See Decision 19a." | |
| exit 1 | |
| fi | |
| python -c "import fastembed, onnxruntime; print('fastembed', fastembed.__version__)" | |
| # Skip the rebuild when nothing indexable changed. The digest covers the | |
| # corpus content plus the chunking and model parameters, but deliberately | |
| # not the git SHA -- most commits touch no indexed file, and keying on the | |
| # SHA would re-upload an identical 18 MB asset every night. | |
| - name: Decide whether the corpus changed | |
| id: digest | |
| env: | |
| ASSET_BASE: https://github.com/${{ github.repository }}/releases/download/dev-wheels | |
| run: | | |
| set -uo pipefail | |
| current="$(aorta chat index digest --public-only --path . | python -c 'import json,sys; print(json.load(sys.stdin)["corpus_digest"])')" | |
| if [ -z "${current}" ]; then | |
| echo "::error::Could not compute the corpus digest." | |
| exit 1 | |
| fi | |
| echo "current=${current}" >> "$GITHUB_OUTPUT" | |
| # A missing or unreadable published manifest means "no baseline", which | |
| # must rebuild rather than skip. Only an explicit digest match skips. | |
| published="" | |
| if curl -fsSL --retry 3 "${ASSET_BASE}/aorta-chat-index.sqlite.manifest.json" -o published.json; then | |
| published="$(python -c 'import json; print(json.load(open("published.json")).get("corpus_digest",""))' || true)" | |
| else | |
| echo "No published manifest yet; building." | |
| fi | |
| # The manifest is the smallest of the three assets and the one most | |
| # likely to survive a partial upload or a manual asset deletion. On | |
| # its own it proves nothing about what `index fetch` will find, so a | |
| # digest match that skips the rebuild would leave fetch broken every | |
| # night until unrelated corpus content happened to change. Require | |
| # the whole published set before trusting the match. | |
| complete=true | |
| for asset in aorta-chat-index.sqlite aorta-chat-index.sqlite.sha256; do | |
| if ! curl -fsSL --retry 3 -I "${ASSET_BASE}/${asset}" >/dev/null; then | |
| echo "Published asset ${asset} is missing; rebuilding the whole set." | |
| complete=false | |
| fi | |
| done | |
| if [ "${complete}" != "true" ]; then | |
| published="" | |
| fi | |
| if [ -n "${published}" ] && [ "${published}" = "${current}" ]; then | |
| echo "changed=false" >> "$GITHUB_OUTPUT" | |
| echo "Corpus digest unchanged (${current}); skipping the rebuild." | |
| else | |
| echo "changed=true" >> "$GITHUB_OUTPUT" | |
| echo "Corpus digest ${published:-<none>} -> ${current}; rebuilding." | |
| fi | |
| # HARD GUARD, part 2 of 2. --public-only re-verifies that origin resolves | |
| # to ROCm/aorta and restricts the corpus to files git tracks. The remote | |
| # check catches a misconfigured workflow; the tracked-file filter catches | |
| # an untracked internal reproducer or customer bundle in the working | |
| # directory, which the remote check cannot see. Both raise, neither warns. | |
| - name: Build the index | |
| if: steps.digest.outputs.changed == 'true' | |
| run: | | |
| mkdir -p index-out | |
| aorta chat index build --public-only --path . \ | |
| --output index-out/aorta-chat-index.sqlite | |
| # Belt and braces on the guard above: assert no chunk came from a path the | |
| # public tree does not track. The build already enforces this, so a hit | |
| # here means the enforcement broke -- which is worth a red run. | |
| - name: Verify no untracked source reached the index | |
| if: steps.digest.outputs.changed == 'true' | |
| run: python scripts/verify_chat_index_sources.py index-out/aorta-chat-index.sqlite | |
| - name: Score retrieval on the shipped question set | |
| if: steps.digest.outputs.changed == 'true' | |
| # Decision 19b. Not a gate: the baseline is young and a threshold picked | |
| # today would be arbitrary. Logged so a regression is visible in the run | |
| # history when someone changes the chunker or the model. | |
| env: | |
| AORTA_CHAT_INDEX_PATH: index-out/aorta-chat-index.sqlite | |
| run: aorta chat index eval || echo "::warning::Retrieval eval failed; index still published." | |
| - name: Publish the rolling main index | |
| if: steps.digest.outputs.changed == 'true' | |
| uses: softprops/action-gh-release@v3 | |
| with: | |
| tag_name: dev-wheels | |
| files: | | |
| index-out/aorta-chat-index.sqlite | |
| index-out/aorta-chat-index.sqlite.manifest.json | |
| index-out/aorta-chat-index.sqlite.sha256 | |
| prerelease: true | |
| make_latest: "false" |