Requirements
Cluster
- Kubernetes with CSI support (tested on k3s and standard distributions).
- At least 3 storage nodes for RF3, or 2 for RF2.
- Low-latency networking between storage nodes; a replication network separate from the management network is recommended.
Storage nodes
| Resource | Requirement |
|---|---|
| CPU | x86-64. The SPDK image must be built for the oldest CPU generation in the cluster. |
| Hugepages | 4 GiB of 2 MiB hugepages per node (the agent reserves them at startup). |
| Drives | NVMe (driver nvme, by PCI address) or block devices (driver uring or aio). |
| Kernel | The nvme-tcp module must be available on nodes that mount volumes. |
:::tip Hypervisors
Some hypervisors (e.g. VMware) block io_uring inside the VM. In that case, use driver: aio for those drives.
:::
Preparing the drives
The drives you use must be empty. If they were used before (another RAID, another file system), wipe the metadata at the start and end of the drive before adding it to a pool:
DISK=/dev/disk/by-id/<disk-id>
SIZE_MB=$(( $(blockdev --getsize64 "$DISK") / 1024 / 1024 ))
dd if=/dev/zero of="$DISK" bs=1M count=2 oflag=direct
dd if=/dev/zero of="$DISK" bs=1M count=2 seek=$(( SIZE_MB - 2 )) oflag=direct
danger
This irreversibly destroys the data on the drive.
Labeling the nodes
The QuasarSDS agent only runs on nodes labeled as storage nodes:
kubectl label node <node> quasarsds.io/storage-node=true