Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 21 additions & 10 deletions docs/architecture/concepts/logical-volumes.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ A Logical Volume (LV) in simplyblock is an abstracted storage entity dynamically
by the simplyblock system. Unlike traditional block storage, simplyblock’s LVs offer advanced features such as thin
provisioning, snapshotting, and replication to enhance resilience and scalability.

A volume is connected to the cluster via NVMe-oF (TCP or RoCEv2).

Key characteristics of Logical Volumes include:

- **Dynamic Allocation:** LVs can be created, resized, and deleted on demand without manual intervention in the
Expand All @@ -21,18 +23,27 @@ Key characteristics of Logical Volumes include:
workloads.
- **Fault Tolerance:** Data is distributed across multiple nodes to prevent data loss and improve reliability.

Two basic types of logical volumes are supported by simplyblock:
Simplyblock has no limit to the capacity of a single volume in relation to the size of the cluster: a single volume
can consume all the cluster capacity, or cluster capacity can be distributed across 50,000 volumes.

Volumes can also contain an almost arbitrary number of snapshots, and new volumes can be created from any snapshot
(copy-on-write cloning).

Simplyblock allows to "tune" the performance and network isolation of volumes and the cardinality between NVMe
namespaces (/dev/nvme1n1, /dev/nvme1n2, ...) and NVMe subsystems (/dev/nvme1n1, /dev/nvme2n1, ...)
can be set on storage-pool level.

- **NVMe-oF Subsystems**: Each logical volume is backed by a separate set of queue pairs. By default, each subsystem
provides three queue parts and one network connection.
NVMe subsystems can be created with a number of namespaces in between one and 500.

Volumes show up in Linux using `lsblk` as `/dev/nvme0n2`, `/dev/nvme1n1`, `/dev/nvmeXn1`, ...
A subsystem comes with its own set of TCP connections. The amount corresponds to the number of queue pairs on the subsystem but
is always limited by the number of cores/vcpus on the client. Meaning, a client with 10 vCPUs cannot connect via more than 10
queue pairs to any NVMe-oF subsystem, even if that subsystem provides 32 or more queue pairs.

Depending on the size of a storage node and particularly its available network bandwidth, the upper recommended limit of
subsystems per node typically lies between five and 50. Therefore, it depends on the upper number of provisioned volumes targeted per node
how to set the relationship between namespaces and subsystems.

- **NVMe-oF Namespaces**: Each logical volume is backed by an NVMe namespace. A namespace is a feature similar to a
logical partition of a drive, although it is defined on the NVMe level (device or target). Up to 32 namespaces share
a single NVMe subsystem and its queue pairs and connections.
If the target is only a handful of high-performance volumes, the number of namespaces can be set to 1. However, if the
target is, for example, 2,000 small volumes, the number of namespaces should be set to a higher value, for example, 100.

This is a more resource-efficient, but performance-limited, version of an individual volume. It is useful, if many,
small volumes are required. Both methods can be combined in a single cluster.

Volumes show up in Linux using `lsblk` as `/dev/nvme0n1`, `/dev/nvme0n2`, `/dev/nvme0nX`, ...
Loading