fix: restore github actions build chain and make multi-arch on native arm64 #21
+322
−128
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wdconinc
force-pushed
the
github-actions-build-chain
branch
from
f6ce24e to
0ae1d32
Compare
wdconinc
force-pushed
the
github-actions-build-chain
branch
from
1b9cb64 to
1a5b4da
Compare
Contributor
There was a problem hiding this comment.
Pull Request Overview
This pull request refactors the GitHub Actions workflow for building and pushing container images, restructuring the job definitions and consolidating build processes.
- Consolidates the base and dev jobs into a single base job that builds the debian_stable_base image with all necessary spack configurations
- Adds support for loading and passing spack-packages configuration as build arguments
- Restructures the eic job (formerly xl) to depend on the base job and use a simplified mirror configuration
- Adds a conditional check to only push to the EICweb mirror on push events
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| containers/debian/Dockerfile | Adds variable preprocessing to convert space-separated cherry-pick lists to newline-separated format for more robust shell processing |
| .github/workflows/mirror.yaml | Restricts mirror push to only occur on push events |
| .github/workflows/build-push.yml | Restructures build jobs by consolidating base/dev jobs, adding spack-packages support, and reorganizing the eic (formerly xl) job configuration |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
wdconinc
commented
Nov 8, 2025
Contributor
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
wdconinc
changed the title
wdconinc
force-pushed
the
github-actions-build-chain
branch
5 times, most recently
from
db4b049 to
d42da52
Compare
veprbl
approved these changes
Nov 11, 2025
Member
veprbl
left a comment
veprbl
left a comment
There was a problem hiding this comment.
Have not looked in detail, but looks reasonable
wdconinc
commented
Nov 11, 2025
Updated the way cherrypicks and cherrypicks_files are echoed to the output, using EOF for multiline support.
Updated the way cherrypicks and cherrypicks_files are echoed to the output in the GitHub Actions workflow.
Updated quotes for SPACK_CHERRYPICKS and SPACKPACKAGES_CHERRYPICKS variables.
Moved registry user and token exports to secrets section.
Co-authored-by: Copilot <[email protected]>
Updated paths for digest files in the build-push workflow.
Added build step ID for Docker build and push actions.
Updated Docker metadata extraction to support multiple registries for the debian_stable_base image.
Refactor GitHub Actions workflow to remove Docker Hub login and streamline image tagging.
- Change cache backend from type=gha to type=registry - Cache images stored in ghcr.io/eic/buildcache - Eliminates evictions from 10GB GHA cache limit - Better cache persistence and reuse across workflows
Use buildkit-cache-dance to persist cache mounts (/ccache, /var/cache/apt, /var/cache/spack) in GitHub Actions cache across ephemeral runners. This is separate from the registry cache which stores image layers: - Registry cache: Unlimited size, stores image layers - Actions cache: 10GB limit, stores cache mount contents Cache key strategy: - Primary key: branch name (e.g., github-actions-build-chain) - Fallback: main branch, then any match for architecture Benefits: - ccache works across ephemeral runners - apt package cache persists - spack buildcache blobs cached - Estimated size: 4-9GB (within 10GB limit) References: - https://docs.docker.com/build/ci/github-actions/cache/#cache-mounts - https://github.com/reproducible-containers/buildkit-cache-dance
Add eic-manifest job that mirrors the base-manifest job functionality for eic images. This job: - Runs after the eic job completes - Downloads digest artifacts from amd64 and arm64 builds - Creates multi-architecture manifest lists - Pushes to configured registries (Docker Hub and GHCR) Also updated digest artifact naming to include ENV (xl) to support multiple environments in the future. The manifest combines the separate architecture builds into a single multi-platform image tag that users can pull.
Match the base job format by writing tags@digest instead of just digest. This ensures the eic-manifest job can properly extract the image name from the digest artifacts, consistent with base-manifest.
wdconinc
force-pushed
the
github-actions-build-chain
branch
from
660f109 to
b42cd19
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request makes significant improvements to the CI/CD workflow for building and publishing containers, focusing on multi-architecture support, manifest management, and build pipeline clarity. The changes refactor the
.github/workflows/build-push.ymlworkflow to better handle builds for bothamd64andarm64, introduce manifest creation and publishing steps, and streamline secret and metadata handling.Multi-architecture build and manifest management:
.github/workflows/build-push.ymlto build bothamd64andarm64images using appropriate runners, and added steps to export and upload image digests for each architecture. [1] [2]base-manifestjob that collects image digests from all architectures, analyzes them, and creates a multi-arch manifest list using Docker Buildx imagetools, tagging the manifest with appropriate metadata.Build pipeline and secrets handling:
eicjob, and streamlined the use of secrets in build steps for both base and eic images. [1] [2]Metadata, permissions, and environment:
contents: readandpackages: write, and clarified environment variable usage in the workflow. [1] [2]Dockerfile improvements:
containers/debian/Dockerfilefor both Spack and Spack-packages, ensuring hashes and file arrays are processed correctly when lists are space-separated or newline-separated. [1] [2]Other workflow improvements:
.github/workflows/mirror.yamlto only push to EICweb on push events, preventing unnecessary syncs on other event types.Let me know if you'd like a deeper walkthrough of any part of the new workflow or Dockerfile logic!