Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions docs/architecture/concepts/erasure-coding.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,19 @@ additional parity fragments, enabling data recovery in the event of node failure

Traditional data redundancy methods, such as replication, require multiple full copies of data, leading to significant
storage overhead. Erasure coding improves upon this by using mathematical algorithms to generate parity fragments,
allowing data reconstruction with fewer overheads.
allowing data reconstruction with lower overhead.

The core principle of erasure coding involves breaking data into **k** data fragments and computing **m** parity
fragments. These **k+m** fragments are distributed across multiple storage nodes. The system can recover lost data using
any **k** available fragments, even if up to **m** fragments are missing or corrupted.

In simplyblock, we support **k** values of 1, 2, and 4, as well as **m** values of 0, 1, and 2. It is important to
note that we use distributed erasure coding, in which every chunk of a stripe is stored on a different node. This
way, we not only protect data from drive loss but also ensure data availability in case one or two nodes are not
operational.

Erasure coding has a number of key characteristics:

- **High Fault Tolerance:** Erasure coding can tolerate multiple node failures while allowing full data recovery.
- **Storage Efficiency:** Compared to replication, erasure coding requires less additional storage to achieve similar levels of redundancy.
- **Computational Overhead:** Encoding and decoding operations involve computational complexity, which may impact performance in latency-sensitive applications.
- **Flexibility:** The parameters **k** and **m** can be adjusted to balance redundancy, performance, and storage overhead.
Loading