aws / aws/sagemaker-python-sdk
Transformer.transform() raises pydantic ValidationError ("tags: extra_forbidden") after submitting the job when tags are set
- Ngôn ngữ chính
- Python
- Star
- 2.3k
- Fork
- 1.3k
- Merge trung bình
- 1 ngày 22 giờ
- Pull request đã merge (30 ngày)
- 35
Mô tả
**PySDK Version**
- [ ] PySDK V2 (2.x)
- [x] PySDK V3 (3.x)
**Describe the bug**
Transformer.transform() raises a pydantic.ValidationError for TransformJob whenever the Transformer was created with non-empty tags. The error is raised after the transform job has already been submitted to SageMaker: transform() builds and submits the CreateTransformJob request successfully, then reconstructs a local sagemaker.core.resources.TransformJob from the request dict but that resource model declares no tags field and sets extra="forbid", so the round-trip fails when the request contains tags.
Another major issue is that there are no tags attached to the created transformation job.
Consequences: the transform job is actually created on SageMaker, but the SDK call raises and the caller never gets the job handle. The failure is independent of the tag value/shape, any non-empty tags triggers it, because the tags field itself is rejected on the resource model.
A related issue, the shape of the tags between `ModelTrainer` and `Transformer` are not the same.
**To reproduce**
This is exactly the construction transform() performs:
```python
from sagemaker.core.resources import TransformJob
TransformJob(transform_job_name="x", tags=[{"key": "team", "value": "ml"}])
```
Real-world trigger:
```python
from sagemaker.core.transformer import Transformer
t = Transformer(model_name="my-model", instance_count=1, instance_type="ml.m5.large",
output_path="s3://bucket/out/", tags=[{"Key": "team", "Value": "ml"}])
t.transform(data="s3://bucket/in/", content_type="text/csv") # job submits, then raises
```
Actual behavior:
```python
pydantic_core._pydantic_core.ValidationError: 1 validation error for TransformJob
tags
Extra inputs are not permitted [type=extra_forbidden, input_value=[{'key': 'team', 'value': 'ml'}], input_type=list]
For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden
```
**Expected behavior**
Transformer.transform() succeeds with tags set and returns the job handle; tags are applied to the transform job.
Root Cause
In [sagemaker-core/src/sagemaker/core/transformer.py](https://github.com/aws/sagemaker-python-sdk/blob/v3.15.1/sagemaker-core/src/sagemaker/core/transformer.py), `Transformer.transform()`:
- [L717](https://github.com/aws/sagemaker-python-sdk/blob/v3.15.1/sagemaker-core/src/sagemaker/core/transformer.py#L717): `transform_request["Tags"] = tags`
- [L405](https://github.com/aws/sagemaker-python-sdk/blob/v3.15.1/sagemaker-core/src/sagemaker/core/transformer.py#L405): `create_transform_job(**request)` - job is submitted
- [L411–L412](https://github.com/aws/sagemaker-python-sdk/blob/v3.15.1/sagemaker-core/src/sagemaker/core/transformer.py#L411):
```python
transformed = transform_util(serialized_request, "CreateTransformJobRequest")
self.latest_transform_job = TransformJob(**transformed) # sagemaker.core.resources.TransformJob
```
`transformed` includes a `tags` key, but `sagemaker.core.resources.TransformJob` has no `tags/Tags` field and `model_config["extra"] == "forbid"` → extra_forbidden.
Suggested fix
Any of: (1) add a `tags` field to `sagemaker.core.resources.TransformJob`; (2) drop `tags` from `transformed` before `TransformJob(**transformed)` at L412; or (3) build the post-submit local object via `TransformJob.get(...)` instead of `TransformJob(**transformed)`.
**Screenshots or logs**
If applicable, add screenshots or logs to help explain your problem.
**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**: sagemaker==3.15.1, sagemaker-core==2.16.0
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**:
- **Framework version**:
- **Python version**:
- **CPU or GPU**:
- **Custom Docker image (Y/N)**:
**Additional context**
Add any other context about the problem here.
Hướng dẫn đóng góp
Hướng nghiên cứu
Đọc sagemaker-core/src/sagemaker/core/transformer.py tại các dòng được trích dẫn về việc xây dựng request và tái tạo sau khi submit, sau đó kiểm tra model sagemaker.core.resources.TransformJob. Tái tạo phần khởi tạo TransformJob được minh họa và lời gọi Transformer.transform() với các tag không rỗng; được xem là hoàn tất khi lời gọi trả về job handle của nó và transform job được tạo có các tag đã cung cấp mà không xảy ra lỗi validation.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- aws, python
- Lĩnh vực
- api, machine-learning
- Loại issue
- Lỗi
- Độ khó
- 3/5
- Thời gian dự kiến
- 1-2 ngày
- Mức độ hoạt động
- Ít trao đổi
- Độ rõ ràng
- Đặc tả rõ ràng
- Mức phù hợp với người mới
- 25/100