[EKS] [request]: Auto Mode - Allow FeatureGate `VolumeAttributesClass` in the EBS CSI Driver
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
**Tell us about your request**
This request is to see whether it would be possible to enable the `VolumeAttributesClass` feature gate in the EBS CSI Driver for EKS Auto Mode.
This feature is already enabled automatically in the EKS managed EBS CSI Driver add-on for standard EKS clusters:
```
# From a pod running v1.40.1-eksbuild.1 of the EKS managed add-on
% k describe pod -n kube-system ebs-csi-controller-64774d9677-72fn6
[...]
csi-provisioner:
Image: 602401143452.dkr.ecr.ap-southeast-2.amazonaws.com/eks/csi-provisioner:v5.2.0-eks-1-32-7
Port:
Host Port:
SeccompProfile: RuntimeDefault
Args:
--timeout=60s
[...]
--feature-gates=VolumeAttributesClass=true
[...]
csi-resizer:
Image: 602401143452.dkr.ecr.ap-southeast-2.amazonaws.com/eks/csi-resizer:v1.13.2-eks-1-32-7
Port:
Host Port:
SeccompProfile: RuntimeDefault
Args:
--timeout=60s
[...]
--feature-gates=VolumeAttributesClass=true
```
If you currently try to follow the steps listed in [tagging.md](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/tagging.md#adding-modifying-and-deleting-tags-of-existing-volumes) for a volume managed by EKS Auto Mode, the existing volumes will not have their tags added/modified by the controller.
**Which service(s) is this request for?**
This is for EKS Auto Mode. More specifically, the EBS CSI Driver used for EKS Auto Mode.
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
The number of volumes which can be running in an EKS cluster varies. If there is a need to add and/or modify the tags for these volumes, it would be cumbersome to do it manually for X number of EBS volumes.
Enabling the `VolumeAttributesClass` feature in the controller would be a nice-to-have for managing resources.
**Are you currently working around this issue?**
Manually tagging the existing volumes.
You could also set the tag at the `StorageClass`-level, but that would only add the tags when the volumes are created -- it does not work for existing volumes.
**Additional context**
For replication:
- deployment.yaml
```
apiVersion: apps/v1
kind: Deployment
metadata:
name: inflate-stateful
spec:
replicas: 1
selector:
matchLabels:
app: inflate-stateful
template:
metadata:
labels:
app: inflate-stateful
spec:
terminationGracePeriodSeconds: 0
nodeSelector:
eks.amazonaws.com/compute-type: auto
containers:
- name: bash
image: public.ecr.aws/docker/library/bash:4.4
command: ["/usr/local/bin/bash"]
args: ["-c", "while true; do echo $(date -u) >> /data/out.txt; sleep 60; done"]
resources:
requests:
cpu: "1"
volumeMounts:
- name: persistent-storage
mountPath: /data
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: auto-ebs-claim
```
- pvc.yaml
```
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: auto-ebs-claim
spec:
accessModes:
- ReadWriteOnce
storageClassName: auto-ebs-sc
resources:
requests:
storage: 8Gi
```
- sc.yaml
```
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: auto-ebs-sc
annotations:
storageclass.kubernetes.io/is-default-class: "true"
provisioner: ebs.csi.eks.amazonaws.com
volumeBindingMode: WaitForFirstConsumer
parameters:
type: gp3
```
- vac.yaml
```
apiVersion: storage.k8s.io/v1beta1
driverName: ebs.csi.eks.amazonaws.com
kind: VolumeAttributesClass
metadata:
name: ebs-daily-backup
parameters:
tagSpecification_1: "Backup-Policy=ebs-daily"
```
Steps in order:
- Apply `sc.yaml`, `pvc.yaml`, `deployment.yaml`, `vac.yaml`
- Describe volume created for the test deployment (`kubectl describe pv ` -> `aws ec2 describe-volumes --volume-ids vol-XXX`)
- Patch PVC to specify the VAC (`kubectl patch pvc auto-ebs-claim -p '{"spec":{"volumeAttributesClassName":"ebs-daily-backup"}}'`)
- Describe volume again to confirm if tags exist
This has been tested on EKS versions 1.31 and 1.32. The CSI Driver version for Auto Mode was `v1.40.1` and `v1.41.0` as found by describing the CSI Driver resource - `kubectl describe csidrivers ebs.csi.eks.amazonaws.com`.
**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)
Contributor guide
Research direction
Start with the deployment.yaml, pvc.yaml, sc.yaml, and vac.yaml replication manifests, then follow the listed kubectl and AWS volume-description steps on EKS Auto Mode. Investigate where the Auto Mode EBS CSI Driver configures controller feature gates; done means patching the PVC with the VolumeAttributesClass causes the existing volume's tags to be added or modified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100