Change Redis brokerUrl

[jeff-cook]

We have installed airflow with helm 1.18.0 using the redis default config. We are looking at switching to an external redis instance so we can get metrics.

However I see this comment that it can't be changed.

data:
  # Note: brokerUrl can only be set during install, not upgrade
  brokerUrl: ~

Is there a process to update the brokerUrl? Example does the database and all the PVCs need to be deleted?

[mkzung]

No need to touch the database or the PVCs. Put the URL in your own Secret and set data.brokerUrlSecretName.

The note is there because <release>-broker-url is a pre-install hook, rendered on install and never on upgrade, so changing data.brokerUrl later does nothing.

kubectl create secret generic airflow-broker-url \
  --from-literal=connection='redis://:PASSWORD@my-redis.example.com:6379/0'

redis:
  enabled: false
data:
  brokerUrlSecretName: airflow-broker-url   # key must be "connection"

With helm template on chart 1.18.0 every AIRFLOW__CELERY__BROKER_URL then reads your secret and the Redis StatefulSet is gone. Helm won't delete the old secret or the Redis PVC, and let the queue drain before switching.