Block Storage Volumes are unattributable in the Cloud Manager, and the obvious fix breaks them
Nobody has claimed this yet.
- Dominant language
- Go Template
- Stars
- 2.3k
- Forks
- 186
- Avg merge
- 3d 11h
- Merged PRs (30d)
- 66
Description
Platform-impact companion to **linode/linode-blockstorage-csi-driver#603**, which carries the technical request. Filing here because APL users hit this, and because the workaround is dangerous enough to be worth naming explicitly.
### The problem for APL operators
Every Block Storage Volume APL provisions is labelled `pvc-` in the Cloud Manager. There is no way to tell which workload owns one without joining back through Kubernetes — and once the cluster is deleted, no way at all. A leaked Volume on a dead cluster is anonymous.
That affects day-to-day cost attribution, the quota census, and cleanup after a failed cluster.
### The trap — please do not "fix" this by renaming
Renaming the Volume to something readable is the obvious workaround and it **permanently breaks the Volume**. The CSI resolves the block device by label, from the label baked into the PV's immutable `volumeHandle`. Rename it and the driver hunts a device path that no longer exists:
```
MountVolume.MountDevice failed for volume "pvc-72af5c8ff02c4813":
Unable to find device path out of attempted paths:
[/dev/disk/by-id/linode-pvc-72af5c8ff02c4813
/dev/disk/by-id/scsi-0Linode_Volume_pvc-72af5c8ff02c4813]
```
Worse, it is a **deferred** failure. A renamed Volume that is already mounted keeps working, and fails on its next attach — a node drain, a reschedule, an upgrade. We ran a reconciler that did exactly this. It reproduced 3/3 on fresh clusters, and on a long-lived one presented as "1 of 3 Loki ingesters ready for 16 days" — a partial outage nobody attributed to storage. We have retired it; there is no safe subset.
If any APL component or downstream distribution renames CSI-provisioned Volumes, it has this bug.
### What APL could do
1. **Enable `--extra-create-metadata`** on the external-provisioner APL ships or depends on. It is the precondition for #603: without it the driver never receives `csi.storage.k8s.io/pvc/name` and `/pvc/namespace`, so the fix cannot work even once merged. On managed LKE-E this is not operator-settable, which is why it needs to come from the platform.
2. **Carry the request upstream** — #603 is small and safe by construction, because `CreateVolume` builds the handle from the Volume it just created, so label and handle cannot diverge.
3. **Consider Volume tags in the meantime.** Tags are not part of any device path, so they can be written and corrected freely. A per-workload tag gives back at-a-glance identity in the Cloud Manager with none of the risk above. This is the direction we are taking downstream.
Happy to supply the reproduction detail, the reconciler logs correlating the rename against the failed mount, or review of a fix.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the external-provisioner configuration APL ships or depends on, then read linode/linode-blockstorage-csi-driver#603 and the documented device-path failure. Confirm the chosen platform direction—metadata support, carrying the upstream request, or Volume tags—against the reproduction details; done means workload identity is recoverable without renaming CSI-provisioned Volumes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100