ceph / ceph/ceph-csi

Optimize RBD mirrors to be space efficient

Open
#5,512 4 comments 0 reactions 0 assignees View on GitHub
component/rbd dependency/ceph keepalive
Dominant language
Go
Stars
1.6k
Forks
617
Avg merge
5d 10h
Merged PRs (30d)
43

Description

# Describe the feature you'd like to have #

RBD mirrors are currently problematic in 2 instances:
- The VolumeReplication is enabled on a volume A, whose parent B is in the trash
- The VolumeReplication is enabled on a volume A, whose parent B is not mirrored (no VolumeReplication on it)
- Usually, in that case, the parent is a VolumeSnapshot, which is (unless I'm wrong), not referenceable in the VolumeReplication CRD

To fix those issues, one can set the flattenMode to "force", which will flatten the RBD image before trying the mirroring process. This will fix any issue, at the cost of extreme space ineffiency.

This is especially true if mirroring all PVCs from a cluster to another. The space efficiency brough by snapshots will be cancelled as every PVC needs to be flattened before being mirrored.

I propose the following:
- If the image is in the trash, flatten it before trying to mirror (or get it out of the trash?)
- If the parent is not mirrored, mirror the parent and then mirror the child (gotta be careful introducing that dependency, as it also requires cleaning up the parent on the destination cluster, and that parent may be referenced by multiple children)

Another possibility for the second proposition is to add a feature to the Ceph-CSI to mirror snapshots.

For example, the specs of a VolumeReplication could be:
```yaml
spec:
dataSource:
apiGroup: ""
kind: VolumeSnapshot
name: abcd
```

instead of the standard `kind: PersistentVolumeClaim`

# What is the value to the end user? (why is it a priority?) #

More space efficiency for users that want to mirror their clusters

# How will we know we have a good solution? (acceptance criteria) #

- Trashed parents are non-blocking
- Missing mirroring on the parents are non-blocking
- Flattening is done only when it can't be done otherwise

# Additional context #

Add any other context or screenshots about the feature request here.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how VolumeReplication handles RBD images, parent images, trash entries, flattenMode, and mirroring dependencies. Compare the proposed parent-mirroring and snapshot-mirroring approaches, including destination cleanup and shared parents. Done means trashed or unmirrored parents do not block replication and flattening occurs only when necessary; the issue does not name files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
distributed-systems, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.