aws / aws/sagemaker-python-sdk

DataCaptureConfigSummary.kms_key_id is required but should be Optional — Endpoint.get() fails

Open
#5,738 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.3k
Forks
1.3k
Avg merge
2d 2h
Merged PRs (30d)
32

Description

**PySDK Version**
- [X] PySDK V3 (3.7.0)

**Describe the bug**

`DataCaptureConfigSummary.kms_key_id` in `sagemaker-core` is defined as a required field, but the SageMaker `DescribeEndpoint` API omits `KmsKeyId` from the response when no customer-managed KMS key is configured. This causes `Endpoint.get()` to fail with a Pydantic `ValidationError` on any endpoint that has data capture enabled without a KMS key.

The input shape `DataCaptureConfig` correctly marks `kms_key_id` as `Optional[StrPipeVar] = Unassigned()`, but the output shape `DataCaptureConfigSummary` has it as required: `kms_key_id: StrPipeVar`.

**To reproduce**

```python
"""
Reproduction: Endpoint.get() fails on endpoints with data capture enabled but no KMS key.

Prerequisites:
pip install sagemaker boto3
An existing SageMaker realtime endpoint with data capture enabled and NO KMS key.
"""
import boto3

session = boto3.Session()
sm = session.client("sagemaker")

# Replace with your endpoint that has data capture enabled (no KMS key)
endpoint_name = "my-endpoint"

# Step 1: Verify the API response omits KmsKeyId
desc = sm.describe_endpoint(EndpointName=endpoint_name)
dc = desc.get("DataCaptureConfig", {})
print(f"EnableCapture: {dc.get('EnableCapture')}") # True
print(f"KmsKeyId present: {'KmsKeyId' in dc}") # False

# Step 2: This fails with ValidationError
from sagemaker.core.resources import Endpoint

endpoint = Endpoint.get(endpoint_name, session=session)
```

**Expected behavior**

`Endpoint.get()` should succeed. `KmsKeyId` is optional in the SageMaker API — endpoints can have data capture enabled without a customer-managed KMS key (S3 uses default encryption in that case).

**Screenshots or logs**

```
ValidationError: 1 validation error for Endpoint
data_capture_config.kms_key_id
Field required [type=missing, input_value={'enable_capture': True,
...'destination_s3_uri': 's3://...'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.12/v/missing
```

**System information**
- **SageMaker Python SDK version**: 3.7.0 (sagemaker-core 2.7.1)
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**: N/A — applies to any endpoint with data capture
- **Framework version**: N/A
- **Python version**: 3.12 / 3.13
- **CPU or GPU**: CPU
- **Custom Docker image (Y/N)**: N

**Additional context**

The fix is a one-line change in the auto-generated `DataCaptureConfigSummary` class in `sagemaker/core/shapes/shapes.py`:

```python
# Current (broken) — required field:
kms_key_id: StrPipeVar

# Fix — optional, matching the input shape DataCaptureConfig:
kms_key_id: Optional[StrPipeVar] = Unassigned()
```

Workaround (monkey-patch at import time):
```python
from sagemaker.core.shapes.shapes import DataCaptureConfigSummary
from sagemaker.core.utils.utils import Unassigned

DataCaptureConfigSummary.model_fields["kms_key_id"].default = Unassigned()
DataCaptureConfigSummary.model_rebuild(force=True)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.