aws / aws/amazon-sagemaker-examples

[Bug Report] 'text/csv; charset=utf-8` is not supported in Sagemaker Pipeline with sklearn and xgboost models

Open
#3,235 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

Hi, I have created a Sagemaker Pipeline Model using an Sklearn model followed by an xgboost model. I followed the instructions [here](https://github.com/aws/amazon-sagemaker-examples/issues/729) to set the 'SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT' environment variable but I'm getting an

`ValueError: Content type text/csv; charset=utf-8` is not supported. error when running the batch transform job on the 2nd (xgboost) container that is following the sklearn container.

My pipeline code looks as following:

```
feature_model = SKLearnModel(
model_data=feature_model_s3_path,
sagemaker_session=sagemaker_session,
role=role,
framework_version="0.23-1",
entry_point = os.path.join(BASE_DIR, "scripts", "sagemaker_feature_transform.py")
)

feature_model.env = {"SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT":"text/csv"}

model = XGBoostModel(
framework_version="1.0-1",
model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts,
sagemaker_session=sagemaker_session,
entry_point= os.path.join(BASE_DIR, "scripts", "sagemaker_xgb_training.py"),
role=role
)

pipeline_model = PipelineModel(
name= "pipeline-model",
role=role,
models=[feature_model, model],
sagemaker_session=sagemaker_session
)

```
As inference output code of container 1 (sklearn) I am using:

```
from sagemaker_containers.beta.framework import ( encoders, worker)

def output_fn(prediction, accept):
if accept == "text/csv":
return worker.Response(encoders.encode(prediction, accept), mimetype=accept)
else:
raise RuntimeException("{} accept type is not supported by this script.".format(accept))
```

As inference input code of container 2 (xgb) I am using:

```
def input_fn(request_body, request_content_type):

if request_content_type == "text/libsvm":
return xgb_encoders.libsvm_to_dmatrix(request_body)
elif request_content_type == "text/csv":
return xgb_encoders.csv_to_dmatrix(request_body)
else:
raise ValueError("Content type {} is not supported.".format(request_content_type))
```
It seems like even though I am forcing the output content type of container 1 to "text/csv", what is arriving in container 2 is an unkown "text/csv; charset=utf-8" format. Any ideas of what I am doing wrong ?

Thank you for your help!

Contributor guide

Open the contributing guide

Research direction

Start with scripts/sagemaker_feature_transform.py and its output_fn, then compare the content type it emits with the input_fn in scripts/sagemaker_xgb_training.py. Reproduce the PipelineModel batch transform flow and trace the content type passed between the two containers. Done means the second container processes the first container's output without the reported ValueError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.