aws / aws/amazon-sagemaker-examples
[Bug Report] 'text/csv; charset=utf-8` is not supported in Sagemaker Pipeline with sklearn and xgboost models
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
Hi, I have created a Sagemaker Pipeline Model using an Sklearn model followed by an xgboost model. I followed the instructions [here](https://github.com/aws/amazon-sagemaker-examples/issues/729) to set the 'SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT' environment variable but I'm getting an
`ValueError: Content type text/csv; charset=utf-8` is not supported. error when running the batch transform job on the 2nd (xgboost) container that is following the sklearn container.
My pipeline code looks as following:
```
feature_model = SKLearnModel(
model_data=feature_model_s3_path,
sagemaker_session=sagemaker_session,
role=role,
framework_version="0.23-1",
entry_point = os.path.join(BASE_DIR, "scripts", "sagemaker_feature_transform.py")
)
feature_model.env = {"SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT":"text/csv"}
model = XGBoostModel(
framework_version="1.0-1",
model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts,
sagemaker_session=sagemaker_session,
entry_point= os.path.join(BASE_DIR, "scripts", "sagemaker_xgb_training.py"),
role=role
)
pipeline_model = PipelineModel(
name= "pipeline-model",
role=role,
models=[feature_model, model],
sagemaker_session=sagemaker_session
)
```
As inference output code of container 1 (sklearn) I am using:
```
from sagemaker_containers.beta.framework import ( encoders, worker)
def output_fn(prediction, accept):
if accept == "text/csv":
return worker.Response(encoders.encode(prediction, accept), mimetype=accept)
else:
raise RuntimeException("{} accept type is not supported by this script.".format(accept))
```
As inference input code of container 2 (xgb) I am using:
```
def input_fn(request_body, request_content_type):
if request_content_type == "text/libsvm":
return xgb_encoders.libsvm_to_dmatrix(request_body)
elif request_content_type == "text/csv":
return xgb_encoders.csv_to_dmatrix(request_body)
else:
raise ValueError("Content type {} is not supported.".format(request_content_type))
```
It seems like even though I am forcing the output content type of container 1 to "text/csv", what is arriving in container 2 is an unkown "text/csv; charset=utf-8" format. Any ideas of what I am doing wrong ?
Thank you for your help!
Contributor guide
Research direction
Start with scripts/sagemaker_feature_transform.py and its output_fn, then compare the content type it emits with the input_fn in scripts/sagemaker_xgb_training.py. Reproduce the PipelineModel batch transform flow and trace the content type passed between the two containers. Done means the second container processes the first container's output without the reported ValueError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100