aws / aws/sagemaker-python-sdk
No DataProcessing in local transform job
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 1.3k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 35
Description
**Describe the bug**
When running `instance_type="local"` the `DataProcessing` is not used, and all the input data is sent to the prediction
In the [_perform_batch_inference](https://github.com/aws/sagemaker-python-sdk/blob/449607275d5de7b94255c49b00368fa30b0f8fbb/src/sagemaker/local/entities.py#L481) function in the `entities.py` file there is no use of the `DataProcessing` key from the `kwargs`. - so the input_data / item is sent _as-is_ without any filtering .
**To reproduce**
```
from sagemaker.model import Model
from sagemaker.local import LocalSession
import boto3
model = Model(
model_data='file://to/my/model_data',
role='MY_ROLE',
image_uri='IMAGE_URI',
sagemaker_session= LocalSession(boto3.Session(region_name='my-region'))
)
transformer = model.transformer(
instance_count=1,
instance_type="local",
strategy="MultiRecord",
assemble_with="Line",
output_path="file://my/output/path",
accept="text/csv",
max_concurrent_transforms=1,
)
transformer.transform(
data="file://path/to/my/data/file",
content_type="text/csv",
split_type="Line",
input_filter="$[4]", # this currently seams not to be working in local mode
join_source="Input",
output_filter="$[0]",
)
transformer.wait()
```
**Expected behavior**
The input csv should be filtered using the `input_filter` value.
Also the the output
**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**:
> sagemaker==2.219.0
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**:
> tested with pyTorch model
- **Python version**:
3.10.14
- **Custom Docker image (Y/N)**:
> i'm using a custom image
Contributor guide
Assessment
This issue has not been assessed yet.