Skip to content

feat: use github releases to decide on when to build#414

Merged
AtzeDeVries merged 52 commits intomainfrom
build-based-on-tags
Apr 22, 2026
Merged

feat: use github releases to decide on when to build#414
AtzeDeVries merged 52 commits intomainfrom
build-based-on-tags

Conversation

@AtzeDeVries
Copy link
Copy Markdown
Contributor

@AtzeDeVries AtzeDeVries commented Dec 10, 2025

This will change the when to build what.

This will do a diff based on a git tag. When a build has been done on main that builds an oci it will set a tag. Next time on main it will do a diff based on this tag. If the diff actually has changes related to this tag.

Why?

We used to do this based on the diff which came with the PR event. This PR event was also used on the main branch. If the job is canceled or removed from queue this creates a problem. Also in github actions, jobs in a queue don't have a guaranteed order. Also rerunning a job based on a PR event would rerun certain steps based on the event data. This is not the behavior you want if the artifact is already published. Also the PR invent was a bit hard to get insight to.

What will happen here

When a PR is created the, a diff between the main branch and the PR branch is made and based on the files in the diff decisions are made on which part of CI are required to run.
When merged, mage will look for the most recent tag based on a tag pattern and use this tag as a diff target. When this results in a diff, mage will run certain CI (mage ci for golang with OCI build) and when finished write a new tag. So if you rerun this job it won't rerun because the diff between the tag and the head is none.
This, in the end will give a way more predictable CI. We can also merge the job queue on the main branch resulting less action minutes spend and not being exposed to the order in which jobs run on the main branch.

How to use this

If set inputs.tag-based-diff is set to true this behavior is enabled. This is disabled by default to ensure a smooth transition. It is strongly encouraged to enable this.

Updating your image tags in helm

When a release is created we can trigger a job to update the OCI tags in your helm values. This is best done via renovate. See examples below.

Relevant issues #356

Comment thread internal/docker/docker.go Fixed
@AtzeDeVries AtzeDeVries force-pushed the build-based-on-tags branch 2 times, most recently from 48ceab1 to 118f263 Compare December 18, 2025 13:24
@AtzeDeVries AtzeDeVries marked this pull request as ready for review December 18, 2025 14:54
@AtzeDeVries
Copy link
Copy Markdown
Contributor Author

This works great in combination with renovate. First of all you need a renovate config which is pretty generic

{
  "baseBranchPatterns": [
    "main"
  ],
  "rebaseWhen": "behind-base-branch",
  "labels": [
    "dependencies",
    "renovate",
    "{{depName}}"
  ],
  "automergeStrategy": "squash",
  "enabledManagers": [
    "helmv3",
    "helm-values"
  ],
  "packageRules": [
    {
      "matchDatasources": [
        "helm"
      ],
      "automerge": true
    },
    {
      "branchPrefix": "helm/dev/",
      "matchManagers": [
        "helm-values"
      ],
      "matchDatasources": [
        "docker"
      ],
      "automerge": true,
      "matchFileNames": [
        "/(^|/)values-dev.yaml$/"
      ],
      "addLabels": [
        "development"
      ],
      "prHourlyLimit": 0
    },
    {
      "branchPrefix": "helm/staging/",
      "matchManagers": [
        "helm-values"
      ],
      "matchDatasources": [
        "docker"
      ],
      "automerge": true,
      "matchFileNames": [
        "/(^|/)values-staging.yaml$/"
      ],
      "addLabels": [
        "staging"
      ],
      "prHourlyLimit": 0
    },
    {
      "branchPrefix": "helm/production/",
      "matchManagers": [
        "helm-values"
      ],
      "matchDatasources": [
        "docker"
      ],
      "automerge": true,
      "matchFileNames": [
        "/(^|/)values-production.yaml$/"
      ],
      "addLabels": [
        "production"
      ],
      "prHourlyLimit": 0
    }
  ],
  "helm-values": {
    "managerFilePatterns": [
      "/(^|/)values(-\\w+)?\\.ya?ml$/"
    ]
  }
}

You also need renovate ci, we trigger this on tag creation, so it will be called after the release is done. Renovate will check GAR for new images, it will create. a changelog, or update an existing PR with the new version (and changelog). In the future we can get rid of this workflow and let renovate server handle this

on:
  push:
    tags:
      - "v*"
  workflow_call:
    inputs:
      logLevel:
        description: "Override default log level"
        required: false
        default: "info"
        type: string
    secrets: {}
  workflow_dispatch:
    inputs:
      logLevel:
        description: "Override default log level"
        required: false
        default: "info"
        type: string
  schedule:
    # At 07:30 AM and 12:30 PM, every day
    - cron: "30 7,12 * * *"
jobs:
  renovate:
    permissions:
      contents: write
      pull-requests: write
      id-token: write
      issues: write
    runs-on: ubuntu-24.04
    steps:
      - name: Checkout
        uses: actions/checkout@v6
      - name: Get token
        id: get_token
        uses: actions/create-github-app-token@v2.2.1
        with:
          app-id: 635546
          private-key: ${{ secrets.RENOVATE_APP_PRIVATE_KEY_PEM }}
          owner: ${{github.repository_owner }}
          repositories: helloworld
          # permission-contents: write
          # permission-pull-requests: write
          # permission-issues: write
      - name: Autenticate with GCP
        id: gcp-auth
        uses: google-github-actions/auth@v3
        with:
          token_format: access_token
          workload_identity_provider: projects/889992792607/locations/global/workloadIdentityPools/github-actions/providers/github-actions-provider
          service_account: gh-ap-helloworld@helloworld-shared-0918.iam.gserviceaccount.com
          create_credentials_file: true
      - name: Run Renovate
        uses: renovatebot/github-action@822441559e94f98b67b82d97ab89fe3003b0a247 # v44.2.0
        env:
          # Repository taken from variable to keep configuration file generic
          RENOVATE_REPOSITORIES: ${{ github.repository }}
          # Onboarding not needed for self-hosted
          RENOVATE_ONBOARDING: "false"
          # Username for GitHub authentication (should match GitHub App name + [bot])
          RENOVATE_USERNAME: "coopnorge-renovate[bot]"
          # Git commit author used, must match GitHub App
          RENOVATE_GIT_AUTHOR: "coopnorge-renovate <121964725+coopnorge-renovate[bot]@users.noreply.github.com>"
          # Use GitHub API to create commits (this allows for signed commits from GitHub App)
          RENOVATE_PLATFORM_COMMIT: "true"
          # Override schedule if set
          RENOVATE_FORCE: "true"
          #RENOVATE_FORCE: ${{ github.event.inputs.overrideSchedule == 'true' && '{''schedule'':null}' || '' }}
          RENOVATE_HOST_RULES: '[{"matchHost":"europe-docker.pkg.dev","token":"${{ steps.gcp-auth.outputs.access_token }}"}]'
          RENOVATE_PR_BODY_TEMPLATE: "{{{header}}}{{{table}}}{{{warnings}}}{{{notes}}}{{{changelogs}}}{{{configDescription}}}{{{controls}}}{{{footer}}}"
          LOG_LEVEL: ${{ inputs.logLevel || 'info' }}
        with:
          configurationFile: .github/renovate.json
          token: ${{ steps.get_token.outputs.token }}

@AtzeDeVries
Copy link
Copy Markdown
Contributor Author

Example PR by renovate

https://github.com/coopnorge/helloworld/pull/3091

Copy link
Copy Markdown
Contributor

@bendiknesbo bendiknesbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this approach, as it fixes a bug where several PR's being merged can lead to some changes not being built a docker-image for.

Comment thread docs/index.md Outdated
Comment thread .github/workflows/reusable-goapp.yaml Outdated
Comment thread internal/git/git.go Outdated
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
defaults for now to false

Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
… release found

Using github release allows us to use naming fo the release to do better
filtering.
Fallback to main if not release helps with onboarding.

Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
@AtzeDeVries AtzeDeVries force-pushed the build-based-on-tags branch from 7f3eb20 to 159f8fe Compare April 15, 2026 13:11
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Comment thread .github/workflows/reusable-goapp.yaml
Comment thread .github/workflows/reusable-goapp.yaml Outdated
Comment thread .github/workflows/reusable-goapp.yaml Outdated
Comment thread internal/git/git.go Outdated
Comment thread internal/git/git.go Outdated
Comment thread internal/github/github.go
Comment thread internal/github/github.go
Comment thread internal/github/github_test.go Outdated
Comment thread internal/github/github_test.go Outdated
Comment thread internal/golang/golang.go
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
@AtzeDeVries
Copy link
Copy Markdown
Contributor Author

@bendiknesbo
Copy link
Copy Markdown
Contributor

example release coopnorge/helloworld@v2026.04.21105450 (release) example renovate pr: coopnorge/helloworld#3365

Another benefit that I had missed, is that this will leverage GitHub's ability to detect when something is deployed (screenshot from https://github.com/coopnorge/helloworld/pull/3363):

Screenshot 2026-04-21 at 15 55 09

Copy link
Copy Markdown
Contributor

@bendiknesbo bendiknesbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks awesome to me :)

@AtzeDeVries AtzeDeVries merged commit ff4b3e2 into main Apr 22, 2026
17 checks passed
@AtzeDeVries AtzeDeVries deleted the build-based-on-tags branch April 22, 2026 06:39
description: A comma separated list of Globs which triggers terraform ci
tag-based-diff:
type: boolean
default: true
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You write that this is disabled by default, but make it enabled by default 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants