aws / aws/aws-cdk

(stepfunctions-tasks): EmrCreateCluster{Props} does not expose the LogEncryptionKmsKeyId request parameter of RunJobFlow

Open
#14,408 5 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/aws-stepfunctions-tasks effort/small feature-request p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

## :question: General Issue

### The Question

Starting with EMR 5.30.0, customers can encrypt EMR logs in S3 using a customer-provided KMS key. This requires some [permission setup](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-debugging.html#emr-plan-debugging-logs-archive), as well as providing the KMS key ID via the [LogEncryptionKmsKeyId](https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#EMR-RunJobFlow-request-LogEncryptionKmsKeyId) parameter to `RunJobFlow`.

At work, I use Step Functions workflows to create, add steps to, and terminate EMR clusters. The workflows use the integrations provided by CDK, e.g. [EmrCreateCluster](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-stepfunctions-tasks.EmrCreateCluster.html). I want to take advantage of the EMR feature and encrypt the EMR logs in S3 using a KMS key I've defined. Even in the latest version of CDK, this doesn't seem possible, because the relevant `RunJobFlow` parameter is not exposed to the corresponding CDK construct.

Is there a plan to add support for this request parameter? Thanks!

### Environment

- **CDK CLI Version:** latest
- **Module Version:** latest
- **Node.js Version:** 12.x (irrelevant)
- **OS:** Amazon Linux 2
- **Language:** Java

### Other information

There are no good (= low-code, quick, non-hacky) workarounds. I would have to either:
* forgo encryption for my EMR logs, which is undesirable from a maintainability POV since all my other S3 data is KMS-encrypted. I'd have to define *just* my EMR logging bucket(s) differently in CDK.
* forgo the StepFunctions integration with EMR for creating clusters, and write a Lambda that calls `RunJobFlow` and passes it the parameter that's missing from CDK. That's extra code to write, test, and own.
* try and hack the Step Functions state machine JSON to "inject" the unsupported param manually in the `RunJobFlow` request.

Contributor guide

Open the contributing guide

Research direction

Start at the EmrCreateCluster construct and compare its request properties with the RunJobFlow API parameter documented for LogEncryptionKmsKeyId. Done means callers can provide that KMS key ID through the Step Functions EMR integration without using a Lambda or manually editing the state machine JSON.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
cloud
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.