Self-contained docker build with ARM64 publishing #3962
Conversation
| RUN ./mill -i copyTo --task 'cli[].base-image.nativeImage' --dest "./docker-out/scala-cli" 1>&2 | ||
|
|
||
| FROM debian:stable-slim | ||
| RUN apt update && apt install build-essential libz-dev clang procps -y |
There was a problem hiding this comment.
Note that I'm not sure if installing those dependencies in a base image is a good idea, or better left to downstream images. Even though I see the appeal of making native and native-image stuff work out of the box..
Main concern is the size
There was a problem hiding this comment.
I mean, we definitely want Scala Native to build nicely with it, so clang is a must.
I'd like to keep them, I think.
Gedochao
left a comment
Gedochao
left a comment
There was a problem hiding this comment.
@tgodzik @zielinsky can you take a look as well? I think it'd be good to have another pair of eyes on this (or two)
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| # Configures this workflow to run every time a change is pushed to the branch called `release`. |
There was a problem hiding this comment.
...you mean like, v* tags and main, right?
| strategy: | ||
| fail-fast: true | ||
| matrix: | ||
| os: ["ubuntu-22.04", "ubuntu-22.04-arm"] |
There was a problem hiding this comment.
| os: ["ubuntu-22.04", "ubuntu-22.04-arm"] | |
| os: ["ubuntu-24.04", "ubuntu-24.04-arm"] |
we're using 24.04 everywhere in the build, I think
| retention-days: 1 | ||
|
|
||
| docker_release_merge: | ||
| runs-on: ubuntu-22.04 |
There was a problem hiding this comment.
| runs-on: ubuntu-22.04 | |
| runs-on: ubuntu-24.04 |
| RUN ./mill -i copyTo --task 'cli[].base-image.nativeImage' --dest "./docker-out/scala-cli" 1>&2 | ||
|
|
||
| FROM debian:stable-slim | ||
| RUN apt update && apt install build-essential libz-dev clang procps -y |
There was a problem hiding this comment.
I mean, we definitely want Scala Native to build nicely with it, so clang is a must.
I'd like to keep them, I think.
Static and mostly static images are in the understanding of GraalVM, as per this doc: https://www.graalvm.org/21.3/reference-manual/native-image/StaticImages/index.html
What particular customisation for Linux x86 did you mean?
Shoot away, I'll try to answer, or at least direct you in the direction of an answer. What's there in this area was initially coded by @alexarchambault (who's also the author of |
tgodzik
left a comment
tgodzik
left a comment
There was a problem hiding this comment.
I don't know much about this, I only built basic docker images. Overall look ok as long as it work 😅
| # This step uses [docker/metadata-action](https://github.com/docker/metadata-action#about) to extract tags and labels that will be applied to the specified image. The `id` "meta" allows the output of this step to be referenced in a subsequent step. The `images` value provides the base name for the tags and labels. | ||
| - name: Extract metadata (tags, labels) for Docker | ||
| id: meta | ||
| uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7 |
| # It uses the `tags` and `labels` parameters to tag and label the image with the output from the "meta" step. | ||
| - name: Build and push Docker image | ||
| id: push | ||
| uses: docker/build-push-action@f2a1d5e99d037542a71f64918e516c093c6f3fc4 |
|
|
||
| # This step generates an artifact attestation for the image, which is an unforgeable statement about where and how it was built. It increases supply chain security for people who consume the image. For more information, see "[AUTOTITLE](/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds)." | ||
| - name: Generate artifact attestation | ||
| uses: actions/attest-build-provenance@v1 |
There was a problem hiding this comment.
The attest-build-provenance documentation says to use version @v3
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} |
There was a problem hiding this comment.
| username: ${{ github.actor }} | |
| password: ${{ secrets.GITHUB_TOKEN }} | |
| username: ${{ env.REGISTRY_LOGIN }} | |
| password: ${{ env.REGISTRY_PASSWORD }} |
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} |
There was a problem hiding this comment.
| username: ${{ github.actor }} | |
| password: ${{ secrets.GITHUB_TOKEN }} | |
| username: ${{ env.REGISTRY_LOGIN }} | |
| password: ${{ env.REGISTRY_PASSWORD }} |
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Log in to the Container registry | ||
| uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1 |
There was a problem hiding this comment.
This is a version from two years ago, shouldn't we bump it up to the latest one?
| uses: docker/setup-buildx-action@v2 | ||
|
|
||
| - name: Log in to the Container registry | ||
| uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1 |
4 participants
Fixes #2132
At the moment, the scala-cli docker image is built using a combination of mill tasks, scripts, and dockerfiles that require externally built binary. What's more, it's only built for x86.
With ARM64 runners on GHA being GA, I propose a simplification - a self-contained multi-stage dockerfile, with a custom Github Workflow that merges the images into a single manifest, meaning that docker will pull the correct image no matter the target platform.
Manifest merging and pushing is a complicated step, and Docker have been amending their docs with the example, which is what this workflow is based on.
I've been using this setup in multiple apps, e.g. Mimalyzer.
There are some aspects of the build I don't really understand – static, mostly static images, customisation for Linux x86, etc.
This proposal attempts to do the simplest possible thing, so that building a docker image is just
docker build . -t VirtusLab/scala-cli.Currently, it only publishes to ghcr.io to test out the workflows without disturbing the main image on docker hub. But additional publishing steps are easily added, of course