[Suggestion] Outstanding PRs by @Ayanda-D

[michaelklishin]

RabbitMQ series

4.2.x

Operating system (distribution) used

Linux

How is RabbitMQ deployed?

Generic binary package

What would you like to suggest for a future version of RabbitMQ?

Moved from #14305 to discuss the contributor opinion on a few PRs by @Ayanda-D:

• rabbitmq-queues: a way to grow a quorum queue member set to a target size #13873
• Extend QQ force shrink operations to use regx for queue names when shrinking queues #15021
• rabbitmq-queues: a command to display a member with highest (commit, log, snapshot) index #14237

[deadtrickster]

Regarding #13873, The very nature of the command gives me that bulk vibe. The concern is if it is intended do be some kind of "emergency" operation (but also if it is not, but a little bit less) the already stressed system will get tons of rpc calls and influx of new members - raft sync, snapshot sync, reelection cycles. Was this tested?

[Ayanda-D]

The very nature of the command gives me that bulk vibe

Not sure I fully get what you mean by "bulk vibe"? We take growing QQs as a once off operation (or rarely executed procedure) - this extension is using the existing "grow ", only difference being it picks the <node> randomly for the user per loop, and grows until <target-size> is reached. The grow operation itself is still the same. Currently, doing this one <node> per time from the CLI is very inefficient and requires node-name awareness (and results in multiple RPC calls from the rabbitmq-queues grow). We dont really mind which nodes the QQ grows to (as long they are healthy and available), all we want is target-quorum-cluster-size to be reached.

[Ayanda-D]

just a few follow up notes regarding these proposed new tools:

#13873 - CMQs had 3 HA-Policies for replicating a queue: all, exactly-n and nodes. The least used nodes policy was partly/mainly due to node-awareness of the policy. Using all, exactly-n (the most used policies from what we've seen) was easy & straight-forward. Single policy and the broker does all the work, without any node-name awareness to manage. Which is what we are trying to somewhat make available with QQs on this change. & as mentioned, nothing's changed much beneath - same grow operation, just added random node selection logic for the caller, on the broker (instead of the CLI). Current grow <node> using hardcoded nodenames is restrictive and slower (just doing grow_to_count N and let the broker do most of the processing work).

#15021 - this just helps in filtering/matching queues to be shrunk - when all queues are not required to force shrink on per vhost or globally.

#14237 - this is to help avoid parsing the quorum-status output to get the same info - and doing comparisons from external tools. Processing on the broker is faster and less data sent between the broker and cli-node (this is just for finer visibility on QQ highest member indices - no further implications on the queue). The quorum-status json output can be used to get the same information, but the exchanged payload between node & cli is big and more processing on the caller.

We don't see these updates lying on any critical path to affect performance - hence low risk? unless folks see otherwise (perhaps #13873 & #15021 to be carried out planned periods). These tools and extensions are just to assist on QQ operational procedures.

[michaelklishin]

I am trying to reach consensus on merging #13873 which seems to have received a more neutral review than the rabbitmq-queues shrink extension.