feat(redis): add syncer HA topology support

README.md

@@ -34,7 +34,7 @@ KubeBlocks add-ons.
 | pulsar | pulsar-bookies-recovery-2.11.2<br>pulsar-bookies-recovery-3.0.2<br>pulsar-bookies-recovery-4.0.6<br>pulsar-bookkeeper-2.11.2<br>pulsar-bookkeeper-3.0.2<br>pulsar-bookkeeper-4.0.6<br>pulsar-broker-2.11.2<br>pulsar-broker-3.0.2<br>pulsar-broker-4.0.6<br>pulsar-proxy-2.11.2<br>pulsar-proxy-3.0.2<br>pulsar-proxy-4.0.6<br>pulsar-zookeeper-2.11.2<br>pulsar-zookeeper-3.0.2 | Apache Pulsar is an open-source, distributed messaging and streaming platform built for the cloud. | cjc7373 caiq1nyu |
 | qdrant | qdrant-1.10.0<br>qdrant-1.13.4<br>qdrant-1.15.4<br>qdrant-1.16.3<br>qdrant-1.17.1<br>qdrant-1.5.0<br>qdrant-1.7.3<br>qdrant-1.8.1<br>qdrant-1.8.4 | High-performance, massive-scale Vector Database for the next generation of AI. | cjc7373 |
 | rabbitmq | rabbitmq-3.10.25<br>rabbitmq-3.11.28<br>rabbitmq-3.12.14<br>rabbitmq-3.13.7<br>rabbitmq-3.8.34<br>rabbitmq-3.9.29<br>rabbitmq-4.0.9<br>rabbitmq-4.1.6<br>rabbitmq-4.2.1 | RabbitMQ is a reliable and mature messaging and streaming broker. | xuriwuyun |
-| redis | redis-5.0.12<br>redis-6.2.14<br>redis-6.2.17<br>redis-6.2.18<br>redis-6.2.19<br>redis-6.2.22<br>redis-7.0.6<br>redis-7.2.10<br>redis-7.2.11<br>redis-7.2.12<br>redis-7.2.14<br>redis-7.2.4<br>redis-7.2.7<br>redis-7.4.2<br>redis-7.4.5<br>redis-7.4.6<br>redis-7.4.7<br>redis-7.4.9<br>redis-8.0.1<br>redis-8.0.3<br>redis-8.0.4<br>redis-8.0.5<br>redis-8.2.1<br>redis-8.2.2<br>redis-8.2.3<br>redis-8.2.6<br>redis-8.4.0<br>redis-8.4.3<br>redis-8.6.3<br>redis-cluster-5.0.12<br>redis-cluster-6.2.14<br>redis-cluster-6.2.17<br>redis-cluster-6.2.18<br>redis-cluster-6.2.19<br>redis-cluster-6.2.22<br>redis-cluster-7.0.6<br>redis-cluster-7.2.10<br>redis-cluster-7.2.11<br>redis-cluster-7.2.12<br>redis-cluster-7.2.14<br>redis-cluster-7.2.4<br>redis-cluster-7.2.7<br>redis-cluster-7.4.2<br>redis-cluster-7.4.5<br>redis-cluster-7.4.6<br>redis-cluster-7.4.7<br>redis-cluster-7.4.9<br>redis-cluster-8.0.1<br>redis-cluster-8.0.3<br>redis-cluster-8.0.4<br>redis-cluster-8.0.5<br>redis-cluster-8.2.1<br>redis-cluster-8.2.2<br>redis-cluster-8.2.3<br>redis-cluster-8.2.6<br>redis-cluster-8.4.0<br>redis-cluster-8.4.3<br>redis-cluster-8.6.3<br>redis-sentinel-5.0.12<br>redis-sentinel-6.2.14<br>redis-sentinel-6.2.17<br>redis-sentinel-6.2.18<br>redis-sentinel-6.2.19<br>redis-sentinel-6.2.22<br>redis-sentinel-7.0.6<br>redis-sentinel-7.2.10<br>redis-sentinel-7.2.11<br>redis-sentinel-7.2.12<br>redis-sentinel-7.2.14<br>redis-sentinel-7.2.4<br>redis-sentinel-7.2.7<br>redis-sentinel-7.4.2<br>redis-sentinel-7.4.5<br>redis-sentinel-7.4.6<br>redis-sentinel-7.4.7<br>redis-sentinel-7.4.9<br>redis-sentinel-8.0.1<br>redis-sentinel-8.0.3<br>redis-sentinel-8.0.4<br>redis-sentinel-8.0.5<br>redis-sentinel-8.2.1<br>redis-sentinel-8.2.2<br>redis-sentinel-8.2.3<br>redis-sentinel-8.2.6<br>redis-sentinel-8.4.0<br>redis-sentinel-8.4.3<br>redis-sentinel-8.6.3<br>redis-twemproxy-0.5.0 | Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps. | wangyelei |
+| redis |  | Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps. | wangyelei |
 | risingwave | risingwave-compactor-v1.0.0<br>risingwave-compute-v1.0.0<br>risingwave-connector-v1.0.0<br>risingwave-frontend-v1.0.0<br>risingwave-meta-v1.0.0 | RisingWave is a distributed SQL streaming database that enables cost-efficient and reliable processing of streaming data. | RisingWave Labs |
 | rocketmq | rocketmq-broker-4.9.6<br>rocketmq-dashboard-2.0.1<br>rocketmq-exporter-0.0.3<br>rocketmq-namesrv-4.9.6 | RocketMQ is a distributed messaging and streaming platform originally developed by Alibaba, featuring high throughput, low latency, and strong reliability for processing millions of messages per second. | kizuna-lek |
 | starrocks-ce | starrocks-ce-be-3.2.2<br>starrocks-ce-be-3.3.0<br>starrocks-ce-fe-3.2.2<br>starrocks-ce-fe-3.3.0 | A Linux Foundation project, is the next-generation data platform designed to make data-intensive real-time analytics fast and easy. | ApeCloud yandongxiao |

addons-cluster/redis/templates/_helpers.tpl

@@ -115,7 +115,17 @@ Define redis ComponentSpec with ComponentDefinition.
   - name: LOAD_BALANCER_ENABLED
     value: "true"
   {{- end }}
+  {{- if or (eq .Values.mode "replication-syncer") (eq .Values.mode "replication-twemproxy") }}
+  - name: KB_MAX_LAG
+    value: {{ .Values.maxLagOnSwitchover | quote }}
+  {{- end }}
   serviceVersion: {{ .Values.version }}
+  {{- if eq .Values.mode "replication-syncer" }}
+  configs:
+  - name: redis-replication-config
+    configMap:
+      name: {{ include "kblib.clusterName" . }}-redis-redis-replication-config
+  {{- end }}
   {{- if and .Values.customSecretName .Values.customSecretNamespace }}
   systemAccounts:
     - name: default
@@ -210,6 +220,8 @@ replication mode: 2
 replicas: 1
 {{- else if eq .Values.mode "replication" }}
 replicas: {{ max .Values.replicas 2 }}
+{{- else if eq .Values.mode "replication-syncer" }}
+replicas: {{ max .Values.replicas 2 }}
 {{- else if eq .Values.mode "replication-twemproxy" }}
 replicas: {{ max .Values.replicas 2 }}
 {{- end }}
@@ -313,6 +325,8 @@ metadata:
     apps.kubeblocks.io/mode: {{ .Values.mode }}
   {{- if and .Values.hostNetworkEnabled (eq .Values.mode "cluster") }}
     kubeblocks.io/host-network: "shard"
+  {{- else if and .Values.hostNetworkEnabled (or (eq .Values.mode "replication-syncer") (eq .Values.mode "replication-twemproxy")) }}
+    kubeblocks.io/host-network: "redis"
   {{- else if .Values.hostNetworkEnabled }}
     kubeblocks.io/host-network: "redis,redis-sentinel"
   {{- end }}
@@ -321,4 +335,4 @@ metadata:
   {{- end }}
 spec:
   terminationPolicy: {{ .Values.extra.terminationPolicy }}
-{{- end }}
+{{- end }}

addons-cluster/redis/templates/cluster.yaml

@@ -23,9 +23,10 @@
   {{- if eq .Values.mode "replication" }}
     {{- include "redis-cluster.componentSpec" . | indent 2 }}
     {{- include "redis-cluster.sentinelComponentSpec" . | indent 2 }}
+  {{- else if eq .Values.mode "replication-syncer" }}
+    {{- include "redis-cluster.componentSpec" . | indent 2 }}
   {{- else if eq .Values.mode "replication-twemproxy" }}
     {{- include "redis-cluster.componentSpec" . | indent 2 }}
-    {{- include "redis-cluster.sentinelComponentSpec" . | indent 2 }}
     {{- include "redis-cluster.twemproxyComponentSpec" . | indent 2 }}
   {{- else }}
     {{- include "redis-cluster.componentSpec" . | indent 2 }}

addons-cluster/redis/values.schema.json

@@ -16,6 +16,7 @@
       "enum": [
         "standalone",
         "replication",
+        "replication-syncer",
         "cluster",
         "replication-twemproxy"
       ]

addons-cluster/redis/values.yaml

@@ -6,19 +6,25 @@
 ##
 version: 7.2.10
 
-## @param mode redis cluster topology mode, standalone, replication and cluster
+## @param mode redis cluster topology mode, standalone, replication, replication-syncer and cluster
 ## standalone: single redis instance
 ## replication: primary-secondary replication with redis sentinel, refer: https://redis.io/docs/management/sentinel
-## replication-twemproxy: primary-secondary replication with sentinel and twemproxy, it is just a demo for how to use twemproxy with redis, not recommended for production
+## replication-syncer: primary-secondary replication with syncer-provided HA and Fake Sentinel endpoint, no redis-sentinel component
+## replication-twemproxy: primary-secondary replication with syncer-provided HA and twemproxy, no redis-sentinel component
 ## cluster: official redis cluster, refer: https://redis.io/docs/reference/cluster-spec.
 mode: replication
 
 ## @param replicas specify replicas of replication
 ## if mode is standalone, replicas should be 1
 ## if mode is replication and replicas is 2, it means 1 primary and 1 secondary
+## if mode is replication-syncer and replicas is 2, it means 1 primary and 1 secondary managed by syncer
 ## if mode is cluster and replicas is 2, it means every shard has 1 primary and 1 secondary, the count of shards is specified by redisCluster.shardCount
 replicas: 2
 
+## @param maxLagOnSwitchover maximum allowed Redis replication offset lag for syncer switchover/failover
+## Default matches Redis Sentinel max_allowed_data_loss semantics: 1MiB.
+maxLagOnSwitchover: "1048576"
+
 ## @param cpu
 ##
 cpu: 0.5

addons/redis/README.md

@@ -1299,7 +1299,7 @@ metadata:
 spec:
   terminationPolicy: Delete
   clusterDef: redis
-  topology: replication-twemproxy  # set topology to standalone
+  topology: replication-twemproxy  # redis HA is managed by syncer; no redis-sentinel component
   componentSpecs:
   - name: redis
     replicas: 2
@@ -1319,23 +1319,6 @@ spec:
           resources:
             requests:
               storage: 20Gi
-  - name: redis-sentinel
-    replicas: 3
-    resources:
-      limits:
-        cpu: "0.2"
-        memory:  "0.2Gi"
-      requests:
-        cpu: "0.2"
-        memory:  "0.2Gi"
-    volumeClaimTemplates:
-      - name: data
-        spec:
-          accessModes:
-            - ReadWriteOnce
-          resources:
-            requests:
-              storage: 20Gi
   - name: redis-twemproxy       # add one componet on provisioniing: twemproxy
     replicas: 3
     resources:
@@ -1345,13 +1328,14 @@ spec:
       requests:
         cpu: "0.2"
         memory: "0.2Gi"
+
 ```
 
 ```bash
 kubectl apply -f examples/redis/cluster-twemproxy.yaml
 ```
 
-A cluster named `redis-twemproxy` will be created with three components, one for Redis (2 replicas), one for Sentinel (3 replicas), and one for twemproxy (3 replicas).
+A cluster named `redis-twemproxy` will be created with two components: Redis (2 replicas, HA managed by syncer) and twemproxy (3 replicas).
 
 ```yaml
 # snippet of cluster.yaml
@@ -1360,10 +1344,9 @@ kind: Cluster
 spec:
   terminationPolicy: Delete
   clusterDef: redis
-  topology: replication-twemproxy  # set topology to standalone
+  topology: replication-twemproxy  # redis HA is managed by syncer
   componentSpecs:
   - name: redis
-  - name: redis-sentinel
   - name: redis-twemproxy       # add one componet on provisioniing: twemproxy
     replicas: 3                 # set the desired number of replicas for twemproxy
     resources: