Skip to content

Commit 4030daa

Browse files
wdconincCopilot
andauthored
fix: restore github actions build chain and make multi-arch on native arm64 (#21)
* fix: restore github actions build chain * Propagate spack-packages to build * Add conditional execution for GitLab sync action * Refactor output handling for cherrypicks in workflow Updated the way cherrypicks and cherrypicks_files are echoed to the output, using EOF for multiline support. * Refactor output handling for cherrypicks in workflow Updated the way cherrypicks and cherrypicks_files are echoed to the output in the GitHub Actions workflow. * Change double quotes to single quotes in YAML Updated quotes for SPACK_CHERRYPICKS and SPACKPACKAGES_CHERRYPICKS variables. * Refactor cherry-pick output formatting in workflow * Fix cherrypick handling in Dockerfile * Remove unnecessary quotes from SPACK variables * Add permissions for contents and packages in workflow * Refactor GitHub Actions to use secrets for registry Moved registry user and token exports to secrets section. * Fix KEY4HEPSPACK_(ORGREPO,VERSION) value * Remove unused docker build-args Co-authored-by: Copilot <[email protected]> * Remove duplicate build arguments Co-authored-by: Copilot <[email protected]> * Change from word count to line count of multiline hash variable Co-authored-by: Copilot <[email protected]> * Refactor build-push workflow for matrix runners * Fix typo in workflow name * Fix indentation in build-push.yml * Update Docker image tags in build-push workflow Remove internal tag from Docker image tags section. * Update runner version for ARM64 build * Export and upload build digest in workflow Add steps to export and upload build digest as artifact * Remove push option from build-push workflow Removed the push option from the build step in the workflow. * Enhance GitHub Actions workflow for multi-arch builds Updated the GitHub Actions workflow to include a base manifest job and improved architecture support for building images. Adjusted Docker metadata extraction and login steps for better efficiency. * Update artifact upload paths to include architecture * Fix typo in runner variable for eic job * Update artifact names and paths in build-push workflow * Update digest file names in build-push workflow * Update caching strategy in build-push workflow * Add command to list contents of /tmp/digests * Fix paths for digest files in build-push.yml Updated paths for digest files in the build-push workflow. * Add commands to list and display digests * Add build step ID to Docker actions Added build step ID for Docker build and push actions. * Modify Docker metadata extraction for debian_stable_base Updated Docker metadata extraction to support multiple registries for the debian_stable_base image. * Update build-push.yml for GitHub Container Registry Refactor GitHub Actions workflow to remove Docker Hub login and streamline image tagging. * Update digest output to use tags instead of images * Refactor image name extraction in build-push.yml * Fix variable name for image in build-push.yml * Add 'push=true' to image outputs in workflow * Remove 'push=true' from outputs in build-push.yml * Fix multi-arch build: use repository name instead of tags for digest push * Switch from GHA cache to registry cache to avoid 10GB limit - Change cache backend from type=gha to type=registry - Cache images stored in ghcr.io/eic/buildcache - Eliminates evictions from 10GB GHA cache limit - Better cache persistence and reuse across workflows * Add buildkit-cache-dance for cache mount persistence Use buildkit-cache-dance to persist cache mounts (/ccache, /var/cache/apt, /var/cache/spack) in GitHub Actions cache across ephemeral runners. This is separate from the registry cache which stores image layers: - Registry cache: Unlimited size, stores image layers - Actions cache: 10GB limit, stores cache mount contents Cache key strategy: - Primary key: branch name (e.g., github-actions-build-chain) - Fallback: main branch, then any match for architecture Benefits: - ccache works across ephemeral runners - apt package cache persists - spack buildcache blobs cached - Estimated size: 4-9GB (within 10GB limit) References: - https://docs.docker.com/build/ci/github-actions/cache/#cache-mounts - https://github.com/reproducible-containers/buildkit-cache-dance * Add eic-manifest job to combine multi-arch eic images Add eic-manifest job that mirrors the base-manifest job functionality for eic images. This job: - Runs after the eic job completes - Downloads digest artifacts from amd64 and arm64 builds - Creates multi-architecture manifest lists - Pushes to configured registries (Docker Hub and GHCR) Also updated digest artifact naming to include ENV (xl) to support multiple environments in the future. The manifest combines the separate architecture builds into a single multi-platform image tag that users can pull. * Fix eic digest format to include tags Match the base job format by writing tags@digest instead of just digest. This ensures the eic-manifest job can properly extract the image name from the digest artifacts, consistent with base-manifest. * Update simsipm version to specific git commit * Add cherry-pick for aarch64 patch in simsipm * Update simsipm version requirement in packages.yaml * Correctly parse image:tag in the digest files when merging manifests * Implement caching to registry by ref name * Fix typo in step name * Update cache references in build-push.yml * Load spack-packages.sh for SPACKPACKAGES_VERSION --------- Co-authored-by: Copilot <[email protected]>
1 parent 01aa935 commit 4030daa

File tree

4 files changed

+322
-128
lines changed

4 files changed

+322
-128
lines changed

0 commit comments

Comments
 (0)