ceph / ceph/ceph-csi

nvmeof: QoS Support for NVMe-oF CSI Driver

Open
#5,694 12 comments 0 reactions 2 assignees Claimed by @nixpanic View on GitHub
component/nvme-of keepalive
Dominant language
Go
Stars
1.6k
Forks
617
Avg merge
5d 10h
Merged PRs (30d)
43

Description

## QoS Support for NVMe-oF CSI Driver

### Overview
Add QoS (Quality of Service) support for NVMe-oF namespaces, allowing users to control IOPS and bandwidth limits both at volume creation and during runtime.

### Proposed Implementation

#### 1. QoS at Volume Creation (StorageClass)
Set initial QoS limits via StorageClass parameters. If omitted, namespaces remain unlimited.

**Example StorageClass:**
```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-nvmeof-standard
provisioner: nvmeof.csi.ceph.com
parameters:
pool: mypool
# Optional QoS parameters
qosRwIopsPerSecond: "10000"
qosRwMegabytesPerSecond: "100"
qosReadMegabytesPerSecond: "150"
qosWriteMegabytesPerSecond: "50"
```

**Implementation:** Modify `ControllerCreateVolume()` to:
- Parse QoS parameters from StorageClass
- Call `ns set_qos` API after namespace creation (if parameters exist)
- Handle missing parameters gracefully (no QoS = unlimited)

#### 2. Runtime QoS Modification (VolumeAttributesClass)
Enable QoS changes on existing volumes without recreation using CSI `ControllerModifyVolume()`.

**Example VolumeAttributesClass:**
```yaml
apiVersion: storage.k8s.io/v1beta1
kind: VolumeAttributesClass
metadata:
name: high-performance
driverName: nvmeof.csi.ceph.com
parameters:
qosRwIopsPerSecond: "50000"
qosRwMegabytesPerSecond: "500"
```
Now you want to hook the pvc with the VAC, you should do:
**Usage:**
```bash
# Apply QoS to existing PVC
kubectl patch pvc my-pvc -p '{"spec":{"volumeAttributesClassName":"high-performance"}}'
```

**Implementation:** Add `ControllerModifyVolume()` RPC to:
- Parse QoS parameters from VolumeAttributesClass
- Call `ns set_qos` via GRPC to the GW
- Deploy `csi-resizer` sidecar to monitor VAC changes

### Supported QoS Parameters
All parameters map directly to NVMe-oF gateway `ns set_qos` command:
- `qosRwIopsPerSecond` - R/W IOPS limit (0 = unlimited)
- `qosRwMegabytesPerSecond` - R/W bandwidth limit
- `qosReadMegabytesPerSecond` - Read bandwidth limit
- `qosWriteMegabytesPerSecond` - Write bandwidth limit

### Requirements
- Kubernetes 1.29+ (for VolumeAttributesClass support)
- CSI spec 1.9.0+ (for ControllerModifyVolume)
- `MODIFY_VOLUME` controller capability
- `csi-resizer` sidecar container

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.