sagemaker: Support retention of variant properties for endpoints
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the feature
As described in [the SageMaker `Endpoint` L2 construct RFC](https://github.com/aws/aws-cdk-rfcs/blob/master/text/0431-sagemaker-l2-endpoint.md#feature-additions):
> [Retention of Variant Properties][retain-variant-properties]: Once an endpoint has been deployed, the desired instance count and desired weight can be dynamically adjusted _per production variant_ without changing the backing endpoint config resource. These changes can either be made automatically via Application Auto Scaling or manually by the customer via the SageMaker UpdateEndpointWeightsAndCapacities operation. After making such changes, by default, when updating a SageMaker endpoint to use a new endpoint config resource (such as when making a CloudFormation change an endpoint config that results in resource replacement), the desired instance count and desired weight is reset to match the new endpoint config resource. To bypass this resetting of variant properties, the `RetainAllVariantProperties` boolean flag was added to the endpoint resource, which when set to true, will not reset these variant properties. In addition to this field, `ExcludeRetainedVariantProperties` was also added to the endpoint resource to allow for selective retention of variant properties (e.g., keeping the desired instance count while resetting the desired weight).
[retain-variant-properties]: https://aws.amazon.com/blogs/machine-learning/configuring-autoscaling-inference-endpoints-in-amazon-sagemaker/
**Please 👍 this issue to help with the prioritization of this feature.**
### Use Case
Changes made to a deployed endpoint's variants' properties (e.g., desired instance count, desired weight), either automatically via Application Auto Scaling or manually by the customer via the SageMaker UpdateEndpointWeightsAndCapacities operation, should conditionally be retained during CloudFormation stack updates to prevent artificial resetting of endpoint state (e.g., scaling down an auto-scaled fleet, shifting traffic away from a manually adjusted set of weights).
### Proposed Solution
As described in [the SageMaker `Endpoint` L2 construct RFC](https://github.com/aws/aws-cdk-rfcs/blob/master/text/0431-sagemaker-l2-endpoint.md#feature-additions):
> As the default behavior is already in place (no retention), adding the functionality should consist of incorporating new attribute(s) on the Endpoint L2 construct's props interface and plumbing it through to the underlying L1 resource definition.
### Other Information
_No response_
### Acknowledgements
- [ ] I _may_ be able to implement this feature request
- [ ] This feature might incur a breaking change
### CDK version used
2.54.0-alpha.0
### Environment details (OS name and version, etc.)
macOS Ventura
Contributor guide
Research direction
Start with the SageMaker Endpoint L2 construct and its props interface, then read the linked RFC's feature-additions section and inspect the underlying L1 resource definition. Done means the retention options can be configured through the L2 construct and are passed through to the endpoint resource without changing the default behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100