Skip to content

Conversation

SimonUnge
Copy link
Collaborator

Proposed Changes

This PR implements automatic membership reconciliation for RabbitMQ streams, a mirror of the existing quorum queue membership reconciliation feature, as discussed in this thread created by @tschuettig

In its current form, it behaves verbatim like the feature for Quorum Queues.

In the discussion, @kjnilsson brought up that "member changes for streams isn't as safe to do as for e.g. quorum queues which means that we're probably not quite ready for any such automation." but I wanted us to have a starting point to work from. It is an opt in feature and uses the same steps you'd manually do using the CLI stream commands. It has the same conditions for growing (and shrinking) as quorum queues - perhaps this in an area where we could add more checks, e.g that all replicas are healthy etc?

If this is the route to go, we could create shared code for QQ and Stream auto reconciliation, but I wanted to keep it separate in this PR to be able to clearly discuss the feature.

Core Implementation

rabbit_stream_periodic_membership_reconciliation.erl - Main gen_server that performs periodic membership reconciliation for streams
rabbit_stream_event_subscriber.erl - Event subscriber that triggers reconciliation on cluster events (node up/down, policy changes)

Configuration

Added stream-specific configuration schema entries in rabbit.schema:

stream.continuous_membership_reconciliation.enabled                                                                                                                                                                                                                                                                                                                                                                                                                                                
stream.continuous_membership_reconciliation.auto_remove                                                                                                                                                                                                                                                                                                                                                                                                                                            
stream.continuous_membership_reconciliation.interval                                                                                                                                                                                                                                                                                                                                                                                                                                               
stream.continuous_membership_reconciliation.trigger_interval                                                                                                                                                                                                                                                                                                                                                                                                                                       
stream.continuous_membership_reconciliation.target_group_size    

Types of Changes

What types of changes does your code introduce to this project?
Put an x in the boxes that apply

  • Bug fix (non-breaking change which fixes issue #NNNN)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause an observable behavior change in existing systems)
  • Documentation improvements (corrections, new content, etc)
  • Cosmetic change (whitespace, formatting, etc)
  • Build system and/or CI

Checklist

Put an x in the boxes that apply.
You can also fill these out after creating the PR.
This is simply a reminder of what we are going to look for before merging your code.

  • Mandatory: I (or my employer/client) have have signed the CA (see https://github.com/rabbitmq/cla)
  • I have read the CONTRIBUTING.md document
  • I have added tests that prove my fix is effective or that my feature works
  • All tests pass locally with my changes
  • If relevant, I have added necessary documentation to https://github.com/rabbitmq/rabbitmq-website
  • If relevant, I have added this change to the first version(s) in release-notes that I expect to introduce it

Further Comments

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution
you did and what alternatives you considered, etc.

@mergify mergify bot added the make label Sep 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant