Add Event Target for Amazon SageMaker Model Building Pipelines
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 71
Description
Update the CDK [Event Targets for Amazon EventBridge](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-events-targets-readme.html) to support support [Amazon SageMaker Model Building Pipelines](https://docs.aws.amazon.com/sagemaker/latest/dg/pipeline-eventbridge.html) as a target.
SageMaker Pipeline [definitions](https://docs.aws.amazon.com/cdk/api/latest/python/aws_cdk.aws_sagemaker/CfnPipeline.html) are already supported in the CDK
### Use Case
There are number of scenarios when triggering a SageMaker pipeline would be useful.
* Scheduled Retraining
* Retraining on Drift Detection
* Batch Inference on new data arriving in S3.
### Proposed Solution
Create a new **SageMakerPipeline** event target, which will be similar to [StepFunctions](https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-events-targets/lib/state-machine.ts).
Would need to support list of `Name`:`Value` [PipelineParameters](https://docs.aws.amazon.com/eventbridge/latest/APIReference/API_Target.html#eventbridge-Type-Target-SageMakerPipelineParameters)
Following is an example adapted from the [sate machine](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-events-targets-readme.html#start-a-stepfunctions-state-machine)
```
import * as iam from '@aws-sdk/aws-iam';
import * as sm from '@aws-cdk/aws-sagemaker';
import * as targets from "@aws-cdk/aws-events-targets";
const rule = new events.Rule(stack, 'Rule', {
schedule: events.Schedule.rate(cdk.Duration.minutes(1)),
});
const role = new iam.Role(stack, 'Role', {
assumedBy: new iam.ServicePrincipal('events.amazonaws.com'),
});
const pipeline = new sm.CfnPipeline(stack, 'Pipeline', {
pipelineName: "TestPipeline",
roleArn: role,
pipelineDefinition:{
"PipelineDefinitionBody": "{\"Version\":\"2020-12-01\",\"Parameters\":[{\"Name\":\"InputDataSource\",\"DefaultValue\":\"\"},{\"Name\":\"InstanceCount\",\"Type\":\"Integer\",\"DefaultValue\":1}],\"Steps\":[{\"Name\":\"Training1\",\"Type\":\"Training\",\"Arguments\":{\"InputDataConfig\":[{\"DataSource\":{\"S3DataSource\":{\"S3Uri\":{\"Get\":\"Parameters.InputDataSource\"}}}}],\"OutputDataConfig\":{\"S3OutputPath\":\"s3://my-s3-bucket/\"},\"ResourceConfig\":{\"InstanceType\":\"ml.m5.large\",\"InstanceCount\":{\"Get\":\"Parameters.InstanceCount\"},\"VolumeSizeInGB\":1024}}}]}"
}
})
rule.addTarget(new targets.SageMakerPipeline(pipeline, {
input: events.RuleTargetInput.fromObject({ InstanceCount: '1' }),
}));
```
* [x] :wave: I may be able to implement this feature request
* [ ] :warning: This feature might incur a breaking change
---
This is a :rocket: Feature Request
Contributor guide
Research direction
Start with packages/@aws-cdk/aws-events-targets/lib/state-machine.ts and the Event Targets README linked in the issue. Compare the proposed SageMakerPipeline target with the StepFunctions target and verify how EventBridge PipelineParameters are represented. Done means the target is available for a SageMaker CfnPipeline and accepts the requested Name/Value parameters.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100