ceph / ceph/ceph-csi

Support for CSIStorageCapacity

Open
#6,039 3 comments 0 reactions 0 assignees View on GitHub
dependency/csi dependency/k8s enhancement keepalive
Dominant language
Go
Stars
1.6k
Forks
617
Avg merge
5d 10h
Merged PRs (30d)
43

Description

# Describe the feature you'd like to have #
I'd like to see support for the CSI GetCapacity call, to allow cluster administrators to easily see how much storage is available for each storageclass/topology combination.

# What is the value to the end user? (why is it a priority?) #
The cluster administrator would gain a simple resource detailing how much storage is availabe for this CSI driver. Additionally, this allows several other mechanisms within Kubernetes to work more efficiently:
- creation of new RBD or FS can take into account how much storage is available for which nodes on a cluster
- cluster administrators may create generic alerting for any and all CSI drivers that are close to running out of capacity, based on the CSIStorageCapacity resources

# How will we know we have a good solution? (acceptance criteria) #
- implement the ControllerGetCapacity function for both RBD and FS controllers
- consider defaulting installation manifests to include settings on the external-provisioner sidecar and CSIDriver to enable this feature (see https://github.com/kubernetes-csi/external-provisioner/tree/release-6.1?tab=readme-ov-file#capacity-support)

There may be some discussion as to the storage that is available, considering the following scenario
- say 300 Gi storage is available, spread over 3 100Gi disks (so 100Gi of effective storage, given 3x replication settings)
- 5 blocks/filesystems are created, each with a limit of 10 Gi, and each filesystems actually uses only 5Gi

Should the driver report 50Gi of available space (considering reserved space as taken) or 75Gi (considering used space as taken). The second case may allow for denser deployments, but can also lead to over-provisioning of storage which means a single additional 75Gi filesystem can fill the ceph cluster, and then cause issues for other filesystems.

Additionally, I'm not sure if replication+total raw storage is something this driver wants to concern itself with.

If multiple storageClasses are defined for a single backing storage cluster (e.g. both a block and fs storageclass for a single set of hdd OSD's), what available capacity should be reported? Raw storage, divided by replication settings, minus in-use or reserved space for both blocks and filesystems on this set of OSD's?

There are choices to be made when implementing this, and the implementation itself is likely to be fairly straightforward. I'd recommend first making these choices, and then implementing this feature.

# Additional context #
- CSI spec: https://github.com/container-storage-interface/spec/blob/master/spec.md#getcapacity
- k8s resource (automatically created by external-provisioner): https://kubernetes.io/docs/reference/kubernetes-api/config-and-storage-resources/csi-storage-capacity-v1/
- settings for external-provisioner required: https://github.com/kubernetes-csi/external-provisioner/tree/release-6.1?tab=readme-ov-file#capacity-support

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the CSI GetCapacity specification and the RBD and CephFS controller entry points for ControllerGetCapacity, then review the installation manifests and the external-provisioner and CSIDriver capacity settings. First resolve how replication, reserved versus used space, topology, and shared storage classes should affect reported capacity. Done means both controllers implement the chosen behavior and the manifests enable the feature as appropriate.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
backend-api-design, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.