aws / aws/aws-cdk

sagemaker: Support retention of variant properties for endpoints

Open
#23,151 0 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/aws-sagemaker effort/small feature-request p3
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the feature

As described in [the SageMaker `Endpoint` L2 construct RFC](https://github.com/aws/aws-cdk-rfcs/blob/master/text/0431-sagemaker-l2-endpoint.md#feature-additions):

> [Retention of Variant Properties][retain-variant-properties]: Once an endpoint has been deployed, the desired instance count and desired weight can be dynamically adjusted _per production variant_ without changing the backing endpoint config resource. These changes can either be made automatically via Application Auto Scaling or manually by the customer via the SageMaker UpdateEndpointWeightsAndCapacities operation. After making such changes, by default, when updating a SageMaker endpoint to use a new endpoint config resource (such as when making a CloudFormation change an endpoint config that results in resource replacement), the desired instance count and desired weight is reset to match the new endpoint config resource. To bypass this resetting of variant properties, the `RetainAllVariantProperties` boolean flag was added to the endpoint resource, which when set to true, will not reset these variant properties. In addition to this field, `ExcludeRetainedVariantProperties` was also added to the endpoint resource to allow for selective retention of variant properties (e.g., keeping the desired instance count while resetting the desired weight).

[retain-variant-properties]: https://aws.amazon.com/blogs/machine-learning/configuring-autoscaling-inference-endpoints-in-amazon-sagemaker/

**Please 👍 this issue to help with the prioritization of this feature.**

### Use Case

Changes made to a deployed endpoint's variants' properties (e.g., desired instance count, desired weight), either automatically via Application Auto Scaling or manually by the customer via the SageMaker UpdateEndpointWeightsAndCapacities operation, should conditionally be retained during CloudFormation stack updates to prevent artificial resetting of endpoint state (e.g., scaling down an auto-scaled fleet, shifting traffic away from a manually adjusted set of weights).

### Proposed Solution

As described in [the SageMaker `Endpoint` L2 construct RFC](https://github.com/aws/aws-cdk-rfcs/blob/master/text/0431-sagemaker-l2-endpoint.md#feature-additions):

> As the default behavior is already in place (no retention), adding the functionality should consist of incorporating new attribute(s) on the Endpoint L2 construct's props interface and plumbing it through to the underlying L1 resource definition.

### Other Information

_No response_

### Acknowledgements

- [ ] I _may_ be able to implement this feature request
- [ ] This feature might incur a breaking change

### CDK version used

2.54.0-alpha.0

### Environment details (OS name and version, etc.)

macOS Ventura

Contributor guide

Open the contributing guide

Research direction

Start with the SageMaker Endpoint L2 construct and its props interface, then read the linked RFC's feature-additions section and inspect the underlying L1 resource definition. Done means the retention options can be configured through the L2 construct and are passed through to the endpoint resource without changing the default behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, infrastructure
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.