Skip to content

ci(os-49): migrate VM release builds to shared runners #1177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jtoelke2 wants to merge 5 commits into from
Closed

ci(os-49): migrate VM release builds to shared runners #1177

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e74ddba
ci(os-49): migrate VM release builds to shared runners
jtoelke2 May 5, 2026
3aef6d5
ci(os-49): support scratch VM release tags
jtoelke2 May 5, 2026
3ed721d
ci(vm): install libcap-ng for libkrun builds
jtoelke2 May 5, 2026
aec0db6
ci(vm): export kernel bundle for macos runtime
jtoelke2 May 5, 2026
d214bd3
ci(vm): fix scratch VM dev release builds
jtoelke2 May 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 118 additions & 44 deletions .github/workflows/release-vm-dev.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,34 @@ name: Release VM Dev
#
# Prerequisites: the vm-dev release must already contain kernel runtime
# tarballs. Run the "Release VM Kernel" workflow first if they are missing.
#
# OS-49: this workflow uses shared runners for VM build/package jobs. It does
# not validate VFIO GPU passthrough; that requires a dedicated VFIO-capable host.

on:
push:
branches: [main]
workflow_dispatch:
inputs:
release_tag:
description: GitHub Release tag to update
required: false
default: vm-dev
type: string

permissions:
contents: write
packages: read

# Serialize with release-vm-kernel.yml — both update the vm-dev release.
# Serialize with release-vm-kernel.yml when both workflows update the same
# release tag.
concurrency:
group: vm-dev-release
group: vm-release-${{ inputs.release_tag || 'vm-dev' }}
cancel-in-progress: false

env:
VM_RELEASE_TAG: ${{ inputs.release_tag || 'vm-dev' }}

defaults:
run:
shell: bash
Expand All @@ -32,7 +45,7 @@ jobs:
# ---------------------------------------------------------------------------
compute-versions:
name: Compute Versions
runs-on: build-amd64
runs-on: linux-amd64-cpu8
timeout-minutes: 5
container:
image: ghcr.io/nvidia/openshell/ci:latest
Expand All @@ -56,14 +69,21 @@ jobs:
id: v
run: |
set -euo pipefail
echo "cargo=$(uv run python tasks/scripts/release.py get-version --cargo)" >> "$GITHUB_OUTPUT"
# Scratch VM tags can look numeric to setuptools_scm, for example
# vm-dev-os49-1177 -> 1177. Only patch Cargo.toml with valid SemVer.
cargo_version="$(uv run python tasks/scripts/release.py get-version --cargo)"
if [[ ! "$cargo_version" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z][0-9A-Za-z.-]*)?(\+[0-9A-Za-z][0-9A-Za-z.-]*)?$ ]]; then
echo "::warning::Computed cargo version '${cargo_version}' is not valid SemVer; leaving Cargo.toml version unchanged."
cargo_version=""
fi
printf 'cargo=%s\n' "$cargo_version" >> "$GITHUB_OUTPUT"

# ---------------------------------------------------------------------------
# Download kernel runtime tarballs from the vm-dev release
# Download kernel runtime tarballs from the configured VM release
# ---------------------------------------------------------------------------
download-kernel-runtime:
name: Download Kernel Runtime
runs-on: build-amd64
runs-on: linux-amd64-cpu8
timeout-minutes: 10
container:
image: ghcr.io/nvidia/openshell/ci:latest
Expand All @@ -82,7 +102,7 @@ jobs:

for platform in linux-aarch64 linux-x86_64 darwin-aarch64; do
echo "Downloading vm-runtime-${platform}.tar.zst..."
gh release download vm-dev \
gh release download "${VM_RELEASE_TAG}" \
--repo "${GITHUB_REPOSITORY}" \
--pattern "vm-runtime-${platform}.tar.zst" \
--dir runtime-artifacts \
Expand All @@ -100,9 +120,9 @@ jobs:
if [ ! -f "$file" ]; then
echo "ERROR: Missing ${file}" >&2
echo "" >&2
echo "The vm-dev release does not have kernel runtime artifacts." >&2
echo "The ${VM_RELEASE_TAG} release does not have kernel runtime artifacts." >&2
echo "Run the 'Release VM Kernel' workflow first:" >&2
echo " gh workflow run release-vm-kernel.yml" >&2
echo " gh workflow run release-vm-kernel.yml -f release_tag=${VM_RELEASE_TAG}" >&2
exit 1
fi
echo "OK: ${file} ($(du -sh "$file" | cut -f1))"
Expand All @@ -125,10 +145,10 @@ jobs:
matrix:
include:
- arch: arm64
runner: build-arm64
runner: linux-arm64-cpu8
guest_arch: aarch64
- arch: amd64
runner: build-amd64
runner: linux-amd64-cpu8
guest_arch: x86_64
runs-on: ${{ matrix.runner }}
timeout-minutes: 30
Expand Down Expand Up @@ -189,12 +209,12 @@ jobs:
matrix:
include:
- arch: arm64
runner: build-arm64
runner: linux-arm64-cpu8
target: aarch64-unknown-linux-gnu
platform: linux-aarch64
guest_arch: aarch64
- arch: amd64
runner: build-amd64
runner: linux-amd64-cpu8
target: x86_64-unknown-linux-gnu
platform: linux-x86_64
guest_arch: x86_64
Expand All @@ -208,7 +228,6 @@ jobs:
options: --privileged
env:
MISE_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SCCACHE_MEMCACHED_ENDPOINT: ${{ vars.SCCACHE_MEMCACHED_ENDPOINT }}
OPENSHELL_IMAGE_TAG: dev
steps:
- uses: actions/checkout@v6
Expand Down Expand Up @@ -315,7 +334,7 @@ jobs:
build-vm-macos:
name: Build VM (macOS)
needs: [compute-versions, download-kernel-runtime, build-rootfs]
runs-on: build-amd64
runs-on: linux-amd64-cpu8
timeout-minutes: 60
container:
image: ghcr.io/nvidia/openshell/ci:latest
Expand All @@ -327,7 +346,6 @@ jobs:
- /var/run/docker.sock:/var/run/docker.sock
env:
MISE_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SCCACHE_MEMCACHED_ENDPOINT: ${{ vars.SCCACHE_MEMCACHED_ENDPOINT }}
steps:
- uses: actions/checkout@v6
with:
Expand All @@ -344,6 +362,8 @@ jobs:

- name: Set up Docker Buildx
uses: ./.github/actions/setup-buildx
with:
driver: local

- name: Install zstd
run: apt-get update && apt-get install -y --no-install-recommends zstd && rm -rf /var/lib/apt/lists/*
Expand Down Expand Up @@ -426,12 +446,12 @@ jobs:
matrix:
include:
- arch: arm64
runner: build-arm64
runner: linux-arm64-cpu8
target: aarch64-unknown-linux-gnu
platform: linux-aarch64
guest_arch: aarch64
- arch: amd64
runner: build-amd64
runner: linux-amd64-cpu8
target: x86_64-unknown-linux-gnu
platform: linux-x86_64
guest_arch: x86_64
Expand All @@ -445,7 +465,6 @@ jobs:
options: --privileged
env:
MISE_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SCCACHE_MEMCACHED_ENDPOINT: ${{ vars.SCCACHE_MEMCACHED_ENDPOINT }}
OPENSHELL_IMAGE_TAG: dev
steps:
- uses: actions/checkout@v6
Expand Down Expand Up @@ -543,13 +562,69 @@ jobs:
path: artifacts/*.tar.gz
retention-days: 5

# ---------------------------------------------------------------------------
# Build Linux ARM64 supervisor bundle for embedding into the macOS VM driver
# ---------------------------------------------------------------------------
build-supervisor-arm64:
name: Build Supervisor Bundle (arm64)
runs-on: linux-arm64-cpu8
timeout-minutes: 30
container:
image: ghcr.io/nvidia/openshell/ci:latest
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
env:
MISE_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
OPENSHELL_IMAGE_TAG: dev
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0

- name: Mark workspace safe for git
run: git config --global --add safe.directory "$GITHUB_WORKSPACE"

- name: Fetch tags
run: git fetch --tags --force

- name: Install tools
run: mise install --locked

- name: Cache Rust target and registry
uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
with:
shared-key: driver-vm-supervisor-arm64
cache-directories: .cache/sccache
cache-targets: "true"

- name: Install zstd
run: apt-get update && apt-get install -y --no-install-recommends zstd && rm -rf /var/lib/apt/lists/*

- name: Build bundled supervisor
run: |
set -euo pipefail
tasks/scripts/vm/build-supervisor-bundle.sh --arch aarch64

- name: sccache stats
if: always()
run: mise x -- sccache --show-stats

- name: Upload supervisor bundle
uses: actions/upload-artifact@v4
with:
name: driver-vm-supervisor-arm64
path: target/vm-runtime-compressed/openshell-sandbox.zst
retention-days: 1
if-no-files-found: error

# ---------------------------------------------------------------------------
# Build openshell-driver-vm binary (macOS ARM64 via osxcross)
# ---------------------------------------------------------------------------
build-driver-vm-macos:
name: Build Driver VM (macOS)
needs: [compute-versions, download-kernel-runtime]
runs-on: build-amd64
needs: [compute-versions, download-kernel-runtime, build-supervisor-arm64]
runs-on: linux-amd64-cpu8
timeout-minutes: 60
container:
image: ghcr.io/nvidia/openshell/ci:latest
Expand All @@ -561,7 +636,6 @@ jobs:
- /var/run/docker.sock:/var/run/docker.sock
env:
MISE_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SCCACHE_MEMCACHED_ENDPOINT: ${{ vars.SCCACHE_MEMCACHED_ENDPOINT }}
steps:
- uses: actions/checkout@v6
with:
Expand All @@ -578,6 +652,8 @@ jobs:

- name: Set up Docker Buildx
uses: ./.github/actions/setup-buildx
with:
driver: local

- name: Install zstd
run: apt-get update && apt-get install -y --no-install-recommends zstd && rm -rf /var/lib/apt/lists/*
Expand Down Expand Up @@ -613,20 +689,17 @@ jobs:
echo "Staged macOS compressed artifacts:"
ls -lah "$COMPRESSED_DIR"

- name: Build bundled supervisor
- name: Download bundled supervisor
uses: actions/download-artifact@v4
with:
name: driver-vm-supervisor-arm64
path: target/vm-runtime-compressed-macos/

- name: Verify bundled supervisor
run: |
set -euo pipefail
docker buildx build \
--file deploy/docker/Dockerfile.images \
--platform linux/arm64 \
--build-arg OPENSHELL_CARGO_VERSION="${{ needs.compute-versions.outputs.cargo_version }}" \
--build-arg OPENSHELL_IMAGE_TAG=dev \
--target supervisor-output \
--output type=local,dest=supervisor-out/ \
.

zstd -19 -T0 -f supervisor-out/openshell-sandbox \
-o "${PWD}/target/vm-runtime-compressed-macos/openshell-sandbox.zst"
test -f target/vm-runtime-compressed-macos/openshell-sandbox.zst
ls -lh target/vm-runtime-compressed-macos/openshell-sandbox.zst

- name: Build macOS binary via Docker (osxcross)
run: |
Expand Down Expand Up @@ -657,7 +730,7 @@ jobs:
retention-days: 5

# ---------------------------------------------------------------------------
# Upload all VM binaries to the vm-dev rolling release
# Upload all VM binaries to the configured VM release
# ---------------------------------------------------------------------------
release-vm-dev:
name: Release VM Dev
Expand All @@ -666,7 +739,7 @@ jobs:
- build-vm-macos
- build-driver-vm-linux
- build-driver-vm-macos
runs-on: build-amd64
runs-on: linux-amd64-cpu8
timeout-minutes: 10
steps:
- uses: actions/checkout@v6
Expand Down Expand Up @@ -708,24 +781,25 @@ jobs:
> vm-binary-checksums-sha256.txt
cat vm-binary-checksums-sha256.txt

- name: Ensure vm-dev tag exists
- name: Ensure VM release tag exists
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git tag -fa vm-dev -m "VM Development Build" "${GITHUB_SHA}"
git push --force origin vm-dev
git tag -fa "${VM_RELEASE_TAG}" -m "VM Development Build" "${GITHUB_SHA}"
git push --force origin "refs/tags/${VM_RELEASE_TAG}"

- name: Prune stale VM binary assets from vm-dev release
- name: Prune stale VM binary assets from VM release
uses: actions/github-script@v7
with:
script: |
const [owner, repo] = process.env.GITHUB_REPOSITORY.split('/');
const tag = process.env.VM_RELEASE_TAG;
let release;
try {
release = await github.rest.repos.getReleaseByTag({ owner, repo, tag: 'vm-dev' });
release = await github.rest.repos.getReleaseByTag({ owner, repo, tag });
} catch (err) {
if (err.status === 404) {
core.info('No existing vm-dev release; will create fresh.');
core.info(`No existing ${tag} release; will create fresh.`);
return;
}
throw err;
Expand All @@ -742,12 +816,12 @@ jobs:
}
}

- name: Upload to vm-dev GitHub Release
- name: Upload to VM GitHub Release
uses: softprops/action-gh-release@v2
with:
name: OpenShell VM Development Build
prerelease: true
tag_name: vm-dev
tag_name: ${{ env.VM_RELEASE_TAG }}
target_commitish: ${{ github.sha }}
body: |
Rolling development build of **openshell-vm** — the MicroVM runtime for OpenShell.
Expand Down
Loading
Loading