aws / aws/containers-roadmap

[EKS] [request]: Auto Mode - Allow FeatureGate `VolumeAttributesClass` in the EBS CSI Driver

Open
#2,603 3 comments 17 reactions 0 assignees View on GitHub
EKS EKS Auto Mode Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**
This request is to see whether it would be possible to enable the `VolumeAttributesClass` feature gate in the EBS CSI Driver for EKS Auto Mode.

This feature is already enabled automatically in the EKS managed EBS CSI Driver add-on for standard EKS clusters:
```
# From a pod running v1.40.1-eksbuild.1 of the EKS managed add-on
% k describe pod -n kube-system ebs-csi-controller-64774d9677-72fn6
[...]
csi-provisioner:
Image: 602401143452.dkr.ecr.ap-southeast-2.amazonaws.com/eks/csi-provisioner:v5.2.0-eks-1-32-7
Port:
Host Port:
SeccompProfile: RuntimeDefault
Args:
--timeout=60s
[...]
--feature-gates=VolumeAttributesClass=true
[...]
csi-resizer:
Image: 602401143452.dkr.ecr.ap-southeast-2.amazonaws.com/eks/csi-resizer:v1.13.2-eks-1-32-7
Port:
Host Port:
SeccompProfile: RuntimeDefault
Args:
--timeout=60s
[...]
--feature-gates=VolumeAttributesClass=true
```

If you currently try to follow the steps listed in [tagging.md](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/tagging.md#adding-modifying-and-deleting-tags-of-existing-volumes) for a volume managed by EKS Auto Mode, the existing volumes will not have their tags added/modified by the controller.

**Which service(s) is this request for?**
This is for EKS Auto Mode. More specifically, the EBS CSI Driver used for EKS Auto Mode.

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
The number of volumes which can be running in an EKS cluster varies. If there is a need to add and/or modify the tags for these volumes, it would be cumbersome to do it manually for X number of EBS volumes.

Enabling the `VolumeAttributesClass` feature in the controller would be a nice-to-have for managing resources.

**Are you currently working around this issue?**
Manually tagging the existing volumes.

You could also set the tag at the `StorageClass`-level, but that would only add the tags when the volumes are created -- it does not work for existing volumes.

**Additional context**
For replication:
- deployment.yaml
```
apiVersion: apps/v1
kind: Deployment
metadata:
name: inflate-stateful
spec:
replicas: 1
selector:
matchLabels:
app: inflate-stateful
template:
metadata:
labels:
app: inflate-stateful
spec:
terminationGracePeriodSeconds: 0
nodeSelector:
eks.amazonaws.com/compute-type: auto
containers:
- name: bash
image: public.ecr.aws/docker/library/bash:4.4
command: ["/usr/local/bin/bash"]
args: ["-c", "while true; do echo $(date -u) >> /data/out.txt; sleep 60; done"]
resources:
requests:
cpu: "1"
volumeMounts:
- name: persistent-storage
mountPath: /data
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: auto-ebs-claim
```

- pvc.yaml
```
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: auto-ebs-claim
spec:
accessModes:
- ReadWriteOnce
storageClassName: auto-ebs-sc
resources:
requests:
storage: 8Gi
```

- sc.yaml
```
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: auto-ebs-sc
annotations:
storageclass.kubernetes.io/is-default-class: "true"
provisioner: ebs.csi.eks.amazonaws.com
volumeBindingMode: WaitForFirstConsumer
parameters:
type: gp3
```
- vac.yaml
```
apiVersion: storage.k8s.io/v1beta1
driverName: ebs.csi.eks.amazonaws.com
kind: VolumeAttributesClass
metadata:
name: ebs-daily-backup
parameters:
tagSpecification_1: "Backup-Policy=ebs-daily"
```

Steps in order:
- Apply `sc.yaml`, `pvc.yaml`, `deployment.yaml`, `vac.yaml`
- Describe volume created for the test deployment (`kubectl describe pv ` -> `aws ec2 describe-volumes --volume-ids vol-XXX`)
- Patch PVC to specify the VAC (`kubectl patch pvc auto-ebs-claim -p '{"spec":{"volumeAttributesClassName":"ebs-daily-backup"}}'`)
- Describe volume again to confirm if tags exist

This has been tested on EKS versions 1.31 and 1.32. The CSI Driver version for Auto Mode was `v1.40.1` and `v1.41.0` as found by describing the CSI Driver resource - `kubectl describe csidrivers ebs.csi.eks.amazonaws.com`.

**Attachments**
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

Contributor guide

Open the contributing guide

Research direction

Start with the deployment.yaml, pvc.yaml, sc.yaml, and vac.yaml replication manifests, then follow the listed kubectl and AWS volume-description steps on EKS Auto Mode. Investigate where the Auto Mode EBS CSI Driver configures controller feature gates; done means patching the PVC with the VolumeAttributesClass causes the existing volume's tags to be added or modified.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes
Domain
cloud, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.