aws / aws/sagemaker-python-sdk

Transformer.transform() raises pydantic ValidationError ("tags: extra_forbidden") after submitting the job when tags are set

已关闭
#6,090 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
2.3k
派生
1.3k
平均合并
1 天 22 小时
30 天内合并 PR
35

描述

**PySDK Version**
- [ ] PySDK V2 (2.x)
- [x] PySDK V3 (3.x)

**Describe the bug**
Transformer.transform() raises a pydantic.ValidationError for TransformJob whenever the Transformer was created with non-empty tags. The error is raised after the transform job has already been submitted to SageMaker: transform() builds and submits the CreateTransformJob request successfully, then reconstructs a local sagemaker.core.resources.TransformJob from the request dict but that resource model declares no tags field and sets extra="forbid", so the round-trip fails when the request contains tags.

Another major issue is that there are no tags attached to the created transformation job.

Consequences: the transform job is actually created on SageMaker, but the SDK call raises and the caller never gets the job handle. The failure is independent of the tag value/shape, any non-empty tags triggers it, because the tags field itself is rejected on the resource model.

A related issue, the shape of the tags between `ModelTrainer` and `Transformer` are not the same.

**To reproduce**
This is exactly the construction transform() performs:
```python
from sagemaker.core.resources import TransformJob
TransformJob(transform_job_name="x", tags=[{"key": "team", "value": "ml"}])
```
Real-world trigger:
```python
from sagemaker.core.transformer import Transformer
t = Transformer(model_name="my-model", instance_count=1, instance_type="ml.m5.large",
output_path="s3://bucket/out/", tags=[{"Key": "team", "Value": "ml"}])
t.transform(data="s3://bucket/in/", content_type="text/csv") # job submits, then raises
```
Actual behavior:
```python
pydantic_core._pydantic_core.ValidationError: 1 validation error for TransformJob
tags
Extra inputs are not permitted [type=extra_forbidden, input_value=[{'key': 'team', 'value': 'ml'}], input_type=list]
For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden
```
**Expected behavior**
Transformer.transform() succeeds with tags set and returns the job handle; tags are applied to the transform job.

Root Cause
In [sagemaker-core/src/sagemaker/core/transformer.py](https://github.com/aws/sagemaker-python-sdk/blob/v3.15.1/sagemaker-core/src/sagemaker/core/transformer.py), `Transformer.transform()`:

- [L717](https://github.com/aws/sagemaker-python-sdk/blob/v3.15.1/sagemaker-core/src/sagemaker/core/transformer.py#L717): `transform_request["Tags"] = tags`
- [L405](https://github.com/aws/sagemaker-python-sdk/blob/v3.15.1/sagemaker-core/src/sagemaker/core/transformer.py#L405): `create_transform_job(**request)` - job is submitted
- [L411–L412](https://github.com/aws/sagemaker-python-sdk/blob/v3.15.1/sagemaker-core/src/sagemaker/core/transformer.py#L411):
```python
transformed = transform_util(serialized_request, "CreateTransformJobRequest")
self.latest_transform_job = TransformJob(**transformed) # sagemaker.core.resources.TransformJob
```
`transformed` includes a `tags` key, but `sagemaker.core.resources.TransformJob` has no `tags/Tags` field and `model_config["extra"] == "forbid"` → extra_forbidden.

Suggested fix
Any of: (1) add a `tags` field to `sagemaker.core.resources.TransformJob`; (2) drop `tags` from `transformed` before `TransformJob(**transformed)` at L412; or (3) build the post-submit local object via `TransformJob.get(...)` instead of `TransformJob(**transformed)`.

**Screenshots or logs**
If applicable, add screenshots or logs to help explain your problem.

**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**: sagemaker==3.15.1, sagemaker-core==2.16.0
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**:
- **Framework version**:
- **Python version**:
- **CPU or GPU**:
- **Custom Docker image (Y/N)**:

**Additional context**
Add any other context about the problem here.

贡献指南

打开贡献指南

调研方向

阅读 sagemaker-core/src/sagemaker/core/transformer.py 中引用的请求构建和提交后重建相关代码行,然后检查 sagemaker.core.resources.TransformJob 模型。使用非空 tags 重现所示的 TransformJob 构造和 Transformer.transform() 调用;当该调用返回其 job handle,且创建的 transform job 在没有验证错误的情况下具有所提供的 tags 时,即表示完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
aws, python
领域
api, machine-learning
Issue 类型
缺陷
难度
3/5
预计耗时
1-2 天
活跃度
冷清
描述清晰度
描述清楚
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。