aws / aws/aws-cdk

aws-cdk-lib.aws_stepfunctions_tasks: Support optimized integration with EMR Serverless

Open
#32,061 1 comment 0 reactions 0 assignees View on GitHub
aws-cdk-lib effort/medium feature-request p3
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the feature

Hi CDK team,

Step function already supported the optimized integration of EMR serverless since late 2023 (see [ref](https://aws.amazon.com/about-aws/whats-new/2023/10/aws-step-functions-optimized-integration-amazon-emr-serverless/)). And we can integrate the EMR serverless this way on AWS console when manually creating the step function. However, the CDK library doesn't have such interface in `aws-cdk-lib.aws_stepfunctions_tasks` module. This forces us to still call the AWS service API to be able to integrate to step function.

### Use Case

We have lots of use cases that orchestrate the EMR serverless job with step function, and we love managing the infra in CDK. The lack of this interface prevents us leveraging the optimized integration approach. Currently we have to build another separate step function to call EMR serverless API in order to submit Spark job, poll the status and return result. And all these CDK code, step function are no longer needed if CDK library can support the optimized integration with EMR serverless.

And there're people looking for the same (see [discussion](https://github.com/aws/aws-cdk/discussions/30501))

### Proposed Solution

_No response_

### Other Information

_No response_

### Acknowledgements

- [X] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### CDK version used

CDK v2 2.166.0

### Environment details (OS name and version, etc.)

Linux

Contributor guide

Open the contributing guide

Research direction

Start in the aws-cdk-lib.aws_stepfunctions_tasks module and review how the requested EMR Serverless optimized integration should be represented. Use the AWS announcement and linked discussion for expected behavior, then verify that the new interface can orchestrate the integration without the separate service API workflow described in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.