Unable to Expunge Mirrored RBD Image from Cloudstack
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.4k
- Avg merge
- 6d 19h
- Merged PRs (30d)
- 32
Description
### problem
I'm running into an issue in Cloudstack where when using ceph RBD as primary storage, and having snapshot-based mirroring in Ceph, I cannot delete and expunge the image without manually disabling mirroring:
```
Request failed. (530)
Failed to expunge the volume Volume {"id":1751,"instanceId":null,"name":"TV-REP-02","uuid":"6ad0914e-9172-45d3-87a0-1f34661b1d3f","volumeType":"DATADISK"} in Primary data store : com.cloud.utils.exception.CloudRuntimeException: com.ceph.rbd.RbdException: Failed to find out if snapshot .mirror.primary.625a7cf1-1946-48a3-8fc3-5a4de98bec29.b8365252-c64b-4862-850b-cb29e333ceb7 is protected - No such file or directory
```
### versions
ACS: 4.22.0.0
KVM: Ubuntu 22.04
Ceph: version 18.2.7 (6b0e988052ec84cf2d4a54ff9bbbc5e720b621ad) reef (stable)
### The steps to reproduce the bug
1. Create a disk image in Cloudstack on the RBD storage pool
2. Enable mirroring on the image in ceph
3. Either define a schedule for snapshots at the pool level, define a snapshot schedule for the image itself, or manually initiate a snapshot of the image in ceph
4. Attempt to delete the image in Cloudstack with the expunge flag enabled (or without, it will just fail to delete it later)
5. Get the error
`Failed to find out if snapshot is protected - No such file or directory`
### What to do about it?
It would be excellent if when trying to delete an RBD image from Cloudstack if the command could be passed through first determine whether or not mirroring is enabled, and if so, disable it prior to attempting to delete it:
```
rbd info /
rbd mirror image disable /
```
And then run the actual delete command.
So for example, if the image ID is `df1e2a8b-00ff-4854-b2d4-f8a2d685fa1e'`:
```
root@ceph:/# rbd info cloudstack/df1e2a8b-00ff-4854-b2d4-f8a2d685fa1e
rbd image 'df1e2a8b-00ff-4854-b2d4-f8a2d685fa1e':
size 25 GiB in 6400 objects
order 22 (4 MiB objects)
snapshot_count: 2
id: 3ad0e7be094cd5
block_name_prefix: rbd_data.3ad0e7be094cd5
format: 2
features: layering, exclusive-lock, object-map, fast-diff, deep-flatten, journaling
op_features:
flags:
create_timestamp: Fri Mar 6 20:14:52 2026
access_timestamp: Fri Mar 6 20:14:52 2026
modify_timestamp: Fri Mar 6 20:14:52 2026
journal: 3ad0e7be094cd5
mirroring state: enabled
mirroring mode: snapshot
mirroring global id: a29acb3e-0d54-4de9-ac62-c98e435462ca
mirroring primary: true
root@ceph:/# rbd mirror image disable cloudstack/df1e2a8b-00ff-4854-b2d4-f8a2d685fa1e
Mirroring disabled
```
It would also be excellent to include an option to enable mirroring on an image when it's created in Cloudstack, but that's a little bit outside of the scope of this issue.
Contributor guide
Research direction
Start by reproducing the delete and expunge flow for Ceph RBD primary storage using the listed CloudStack, KVM, and Ceph versions. Trace how mirroring state and protected snapshots are handled before deletion, then verify that an RBD image with snapshot-based mirroring can be deleted and expunged without the reported missing-snapshot error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100