awslabs / awslabs/data-solutions-framework-on-aws
Enhancement in SparkJob: Use the optimized integration between Step Functions and EMR
- Dominant language
- TypeScript
- Stars
- 147
- Forks
- 31
- PR merge metrics
- No merged PRs in 30d
Description
In the SparkJob construct, we could use the optimized integration with EMR Serverless and EMR on EKS instead of the sdk integration.
```[tasklist]
### Tasks
- [ ] [Optimized integration with EMR on EKS](https://docs.aws.amazon.com/step-functions/latest/dg/connect-emr-eks.html)
- [ ] [Optimized integration with EMR Serverless](https://docs.aws.amazon.com/step-functions/latest/dg/connect-emr-serverless.html)
```
Contributor guide
Research direction
Start by locating the SparkJob construct and its current SDK integration. Read the linked AWS Step Functions documentation for optimized integrations with EMR on EKS and EMR Serverless, then verify both task-list items are supported by the construct and covered by the existing project checks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100