Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 16 additions & 9 deletions docs/architecture/concepts/automatic-rebalancing.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,24 @@ weight: 30700
---

Automatic rebalancing is a fundamental feature of distributed data storage systems designed to maintain an even
distribution of data across storage nodes. This process ensures optimal performance, prevents resource overutilization,
distribution of data and performance across storage nodes. This process ensures optimal performance, prevents resource under-utilization,
and enhances system resilience by dynamically redistributing data in response to changes in cluster topology or workload
patterns.

In a distributed storage system, data is typically spread across multiple storage nodes for redundancy, scalability, and
performance. Over time, various factors can lead to an imbalance in data distribution, such as:
In Simplyblock, re-balancing concerns the re-balancing of back storage (for rebalancing of front storage or volume "docking points", see
[volume migration](volume-migration.md) for details). It's behaviour depends on weather local node affinity is turned on.

- The addition of new storage nodes, which initially lack any data.
- The removal or failure of existing nodes, requiring data redistribution to maintain availability.
- The equal distribution of data across storage nodes.
If local node affinity is turned on, primary data chunks are re-balanced with a preference to the node at which also the front storage resides.
If local node affinity is turned off, all data and parity chunks are distributed with the target to achieve absolute equal (in a relative sense) utilization
levels across all NVMe devices in the cluster under consideration of failure domains and data protection rules.

Automatic rebalancing addresses these issues by dynamically redistributing data across the cluster. This process is
driven by an algorithm that continuously monitors data distribution and redistributes data when imbalances are detected.
The goal is to achieve uniform data placement while minimizing performance overhead during the rebalancing process.
Data re-balancing uses three important principles:
- always try to move the longest contigous segments of data to minimize random-access IOPS
- do not use more than 20% of the cluster performance capacity, this is guaranteed by internal QoS
- use maximum parallelism (during migration, load all devices in the cluster equally to maximize migration speed within the 20%)

Rebalancing is used in the following scenarios:

- After temporary node outages (planned or unplanned)
- When expanding the cluster (adding nodes / devices)
- When "failing" devices or entire nodes (rebuild to restablish full redundancy without the devices)
Loading