awslabs / awslabs/data-on-eks

[feature] Spark operator should use Pod Identity

Open
#671 3 comments 1 reaction 0 assignees View on GitHub
data-on-eks enhancement
Dominant language
Shell
Stars
857
Forks
303
Avg merge
11h 5m
Merged PRs (30d)
3

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

#### What is the outcome that you are trying to reach?

EKS has released Pod Identities to help manage IAM rights for pods https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html
The Spark operator solution still uses IRSA but it would be great to have it use Pod Identity instead

Contributor guide

Open the contributing guide

Research direction

Start by reading the EKS Pod Identities documentation linked in the issue, then inspect how the Spark operator currently uses IRSA. Update the operator’s identity configuration to use Pod Identity instead, and verify that the Spark operator can obtain the required IAM permissions without IRSA.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes, spark
Domain
cloud, data-engineering, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.