feat: use github releases to decide on when to build#414
Conversation
282a788 to
6e53531
Compare
1f40310 to
3cec9c5
Compare
48ceab1 to
118f263
Compare
|
This works great in combination with renovate. First of all you need a renovate config which is pretty generic {
"baseBranchPatterns": [
"main"
],
"rebaseWhen": "behind-base-branch",
"labels": [
"dependencies",
"renovate",
"{{depName}}"
],
"automergeStrategy": "squash",
"enabledManagers": [
"helmv3",
"helm-values"
],
"packageRules": [
{
"matchDatasources": [
"helm"
],
"automerge": true
},
{
"branchPrefix": "helm/dev/",
"matchManagers": [
"helm-values"
],
"matchDatasources": [
"docker"
],
"automerge": true,
"matchFileNames": [
"/(^|/)values-dev.yaml$/"
],
"addLabels": [
"development"
],
"prHourlyLimit": 0
},
{
"branchPrefix": "helm/staging/",
"matchManagers": [
"helm-values"
],
"matchDatasources": [
"docker"
],
"automerge": true,
"matchFileNames": [
"/(^|/)values-staging.yaml$/"
],
"addLabels": [
"staging"
],
"prHourlyLimit": 0
},
{
"branchPrefix": "helm/production/",
"matchManagers": [
"helm-values"
],
"matchDatasources": [
"docker"
],
"automerge": true,
"matchFileNames": [
"/(^|/)values-production.yaml$/"
],
"addLabels": [
"production"
],
"prHourlyLimit": 0
}
],
"helm-values": {
"managerFilePatterns": [
"/(^|/)values(-\\w+)?\\.ya?ml$/"
]
}
}You also need renovate ci, we trigger this on tag creation, so it will be called after the release is done. Renovate will check GAR for new images, it will create. a changelog, or update an existing PR with the new version (and changelog). In the future we can get rid of this workflow and let renovate server handle this on:
push:
tags:
- "v*"
workflow_call:
inputs:
logLevel:
description: "Override default log level"
required: false
default: "info"
type: string
secrets: {}
workflow_dispatch:
inputs:
logLevel:
description: "Override default log level"
required: false
default: "info"
type: string
schedule:
# At 07:30 AM and 12:30 PM, every day
- cron: "30 7,12 * * *"
jobs:
renovate:
permissions:
contents: write
pull-requests: write
id-token: write
issues: write
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Get token
id: get_token
uses: actions/create-github-app-token@v2.2.1
with:
app-id: 635546
private-key: ${{ secrets.RENOVATE_APP_PRIVATE_KEY_PEM }}
owner: ${{github.repository_owner }}
repositories: helloworld
# permission-contents: write
# permission-pull-requests: write
# permission-issues: write
- name: Autenticate with GCP
id: gcp-auth
uses: google-github-actions/auth@v3
with:
token_format: access_token
workload_identity_provider: projects/889992792607/locations/global/workloadIdentityPools/github-actions/providers/github-actions-provider
service_account: gh-ap-helloworld@helloworld-shared-0918.iam.gserviceaccount.com
create_credentials_file: true
- name: Run Renovate
uses: renovatebot/github-action@822441559e94f98b67b82d97ab89fe3003b0a247 # v44.2.0
env:
# Repository taken from variable to keep configuration file generic
RENOVATE_REPOSITORIES: ${{ github.repository }}
# Onboarding not needed for self-hosted
RENOVATE_ONBOARDING: "false"
# Username for GitHub authentication (should match GitHub App name + [bot])
RENOVATE_USERNAME: "coopnorge-renovate[bot]"
# Git commit author used, must match GitHub App
RENOVATE_GIT_AUTHOR: "coopnorge-renovate <121964725+coopnorge-renovate[bot]@users.noreply.github.com>"
# Use GitHub API to create commits (this allows for signed commits from GitHub App)
RENOVATE_PLATFORM_COMMIT: "true"
# Override schedule if set
RENOVATE_FORCE: "true"
#RENOVATE_FORCE: ${{ github.event.inputs.overrideSchedule == 'true' && '{''schedule'':null}' || '' }}
RENOVATE_HOST_RULES: '[{"matchHost":"europe-docker.pkg.dev","token":"${{ steps.gcp-auth.outputs.access_token }}"}]'
RENOVATE_PR_BODY_TEMPLATE: "{{{header}}}{{{table}}}{{{warnings}}}{{{notes}}}{{{changelogs}}}{{{configDescription}}}{{{controls}}}{{{footer}}}"
LOG_LEVEL: ${{ inputs.logLevel || 'info' }}
with:
configurationFile: .github/renovate.json
token: ${{ steps.get_token.outputs.token }} |
|
Example PR by renovate |
e32472c to
a70222e
Compare
bendiknesbo
left a comment
There was a problem hiding this comment.
I like this approach, as it fixes a bug where several PR's being merged can lead to some changes not being built a docker-image for.
ecea7c0 to
7cbfedc
Compare
7cbfedc to
83659de
Compare
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
defaults for now to false Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
… release found Using github release allows us to use naming fo the release to do better filtering. Fallback to main if not release helps with onboarding. Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
7f3eb20 to
159f8fe
Compare
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
Signed-off-by: Atze de Vries <atze.wiebe.de.vries@coop.no>
|
example release |
Another benefit that I had missed, is that this will leverage GitHub's ability to detect when something is deployed (screenshot from https://github.com/coopnorge/helloworld/pull/3363):
|
bendiknesbo
left a comment
There was a problem hiding this comment.
This looks awesome to me :)
| description: A comma separated list of Globs which triggers terraform ci | ||
| tag-based-diff: | ||
| type: boolean | ||
| default: true |
There was a problem hiding this comment.
You write that this is disabled by default, but make it enabled by default 😅

This will change the when to build what.
This will do a diff based on a git tag. When a build has been done on main that builds an oci it will set a tag. Next time on main it will do a diff based on this tag. If the diff actually has changes related to this tag.
Why?
We used to do this based on the diff which came with the PR event. This PR event was also used on the main branch. If the job is canceled or removed from queue this creates a problem. Also in github actions, jobs in a queue don't have a guaranteed order. Also rerunning a job based on a PR event would rerun certain steps based on the event data. This is not the behavior you want if the artifact is already published. Also the PR invent was a bit hard to get insight to.
What will happen here
When a PR is created the, a diff between the main branch and the PR branch is made and based on the files in the diff decisions are made on which part of CI are required to run.
When merged, mage will look for the most recent tag based on a tag pattern and use this tag as a diff target. When this results in a diff, mage will run certain CI (mage ci for golang with OCI build) and when finished write a new tag. So if you rerun this job it won't rerun because the diff between the tag and the head is none.
This, in the end will give a way more predictable CI. We can also merge the job queue on the main branch resulting less action minutes spend and not being exposed to the order in which jobs run on the main branch.
How to use this
If set
inputs.tag-based-diffis set totruethis behavior is enabled. This is disabled by default to ensure a smooth transition. It is strongly encouraged to enable this.Updating your image tags in helm
When a release is created we can trigger a job to update the OCI tags in your helm values. This is best done via renovate. See examples below.
Relevant issues #356