stepfunctions-tasks: EmrCreateClusterProps clusterRole should be instance profile not IRole
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
Seems that the API for AWS CDK with AWS Step Function to create an EMR cluster is wrong.
With an EMR service object, we can provide `RunJobFlowInput` to `runJobFlow`, that specified `JobFlowRole: instanceProfile.ref`, with `instanceProfile` as a profile with a single role:
```
// Example role, our role is constructed with more permissions
const myRole = new iam.Role(stack, 'my-role', {
assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com'),
managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AmazonElasticMapReduceforEC2Role')],
});
const instanceProfile = new iam.CfnInstanceProfile(stack, `emr-ec2-instance-profile`, {
roles: [myRole.roleName],
});
```
With `stepfunctionsTasks.EmrCreateCluster`, however, the required `EmrCreateClusterProps`, have renamed `JobFlowRole` into `clusterRole`, and [expect](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-stepfunctions-tasks.EmrCreateClusterProps.html) `clusterRole` to be of type `iam.IRole` instead of a reference to the previous `iam.CfnInstanceProfile`. Attempting to pass the IAM Role that the profile was created with (e.g. `myRole`), results in an `Invalid InstanceProfile` error when running Step Function.
This seems to indicate that despite trying to pass a `IRole`, what the API should be requesting is the reference to a `CfnInstanceProfile`.
### Reproduction Steps
```
// Example role, our role is constructed with more permissions
const myRole = new iam.Role(stack, 'my-role', {
assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com'),
managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AmazonElasticMapReduceforEC2Role')],
});
const instanceProfile = new iam.CfnInstanceProfile(stack, `emr-ec2-instance-profile`, {
roles: [myRole.roleName],
});
const startClusterTask = new steps.Task(stack, 'Create Cluster', {
task: new tasks.EmrCreateCluster({
name: 'myCluster',
visibleToAllUsers: true,
releaseLabel: 'emr-6.0.0',
instances: {
masterInstanceType: 'm5.xlarge',
slaveInstanceType: 'm5.xlarge',
instanceCount: 3,
},
clusterRole: myRole, // aka JowFlowRole or instanceProfile.ref
configurations: [
{
classification: 'spark-env',
properties: {
PYSPARK_PYTHON: '/usr/bin/python3',
},
},
],
applications: [
{
name: 'Spark',
},
],
}),
});
const stopClusterTask = new steps.Task(stack, 'Task', {
task: new tasks.EmrTerminateCluster({
clusterId: 'myCluster',
}),
});
const workflow = steps.Chain.start(startClusterTask).next(stopClusterTask);
const stateMachine = new steps.StateMachine(stack, 'myStateMachine', {
stateMachineType: steps.StateMachineType.STANDARD,
definition: workflow,
});
```
### Error Log
Attempting to run the step function will yield:
```
{
"error": "EMR.AmazonElasticMapReduceException",
"cause": "Invalid InstanceProfile: ${profile}. (Service: AmazonElasticMapReduce; Status Code: 400; Error Code: ValidationException; Request ID: ${Request})"
}
```
### Environment
- **CLI Version :** aws-cli/1.16.227
- **Framework Version:** 1.37.0
- **OS :** MacOS Catalina 10.15.4
- **Language :** Typescript
### Other
The type of `clusterRole` in `EmrCreateClusterProps` should not be `IRole`, but a reference to the instance profile, as expected in the [EMR specification](https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#EMR-RunJobFlow-request-JobFlowRole).
---
This is :bug: Bug Report
Contributor guide
Research direction
Start with the stepfunctions-tasks EmrCreateClusterProps and EmrCreateCluster entry points, then compare clusterRole with EMR RunJobFlow's JobFlowRole and the instanceProfile.ref shown in the reproduction. Reproduce the Invalid InstanceProfile failure and verify that the generated request uses an instance-profile reference; the issue is done when the task accepts and sends the expected value.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100