aws / aws/containers-roadmap

[EKS] [request]: EMR as an addon on EKS

Open
#2,774 2 comments 0 reactions 0 assignees View on GitHub
EKS EKS Add-Ons EMR Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**
We would like to request EMR as a native add-on for EKS clusters.

The goal is to run Spark workloads directly on EKS while keeping cluster management centralized and fully aligned with our Kubernetes-first and GitOps operating model.

As AWS continues to evolve toward an application-first cluster experience, having EMR delivered as a managed EKS add-on would significantly simplify adoption and operational management.

**Which service(s) is this request for?**
EKS, EMR

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
Today, EMR on EKS requires registering a virtual cluster and managing additional configuration outside the standard EKS lifecycle.

This introduces extra setup steps and operational overhead.

If EMR were available as an EKS add-on, it could be enabled directly during cluster creation (via Terraform, eksctl, or the AWS Console), simplifying provisioning and management.

An add-on model would help us:

1. Align fully with our Kubernetes-first architecture
2. Reduce operational complexity compared to standalone EMR clusters
3. Simplify the current EMR on EKS registration workflow
4. Standardize environment setup across dev, staging, and prod
5. Enable centralized logging and metrics integration with CloudWatch by default

**Are you currently working around this issue?**
We are currently using the registered EMR on EKS model and managing virtual cluster setup separately. This requires manual registration steps and additional IAM configuration.

We are aligning Spark workloads with our GitOps and namespace-based environment strategy, but the current setup introduces extra friction during provisioning and onboarding.

An add-on model would streamline deployment, reduce setup time, and make EMR workloads feel like a native part of the EKS cluster lifecycle.

Contributor guide

Open the contributing guide

Research direction

This is a roadmap request for a native EMR add-on on EKS, with Terraform, eksctl, and the AWS Console named as provisioning paths. No implementation files, entry points, or tests are identified; first clarify the add-on design and repository scope with maintainers. Done would require an agreed implementation and validated EMR registration, IAM, and cluster-lifecycle behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes, terraform
Domain
cloud, devops, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.