aws / aws/sagemaker-scikit-learn-container

Sagemaker sklearn custom code “ValueError: could not convert string to float”

Open
#70 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
188
Forks
121
PR merge metrics
No merged PRs in 30d

Description

I am using a sklearn custom script to train and deploy a model in sagemaker. When I tried to invoke the endpoint I had the following error:

`ERROR - model_featurizer_training - Exception on /invocations [POST]
Traceback (most recent call last):
File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper
return fn(*args, **kwargs)
File "/miniconda3/lib/python3.7/site-packages/sagemaker_sklearn_container/serving.py", line 60, in default_input_fn
return np_array.astype(np.float32) if content_type in content_types.UTF8_TYPES else np_array
ValueError: could not convert string to float: 'female'`

My training custom script is as follows:

```
if __name__=='__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--n_estimators', type=int, default=10)

parser.add_argument('--output-data-dir', type=str, default=os.environ['SM_OUTPUT_DATA_DIR'])
parser.add_argument('--model-dir', type=str, default=os.environ['SM_MODEL_DIR'])
parser.add_argument('--train', type=str, default=os.environ['SM_CHANNEL_TRAIN'])

args = parser.parse_args()

input_file = [os.path.join(args.train, file) for file in os.listdir(args.train)]

raw_data = [pd.read_csv(file, engine='python') for file in input_file]

train_data = pd.concat(raw_data)

X = train_data.iloc[:, 1:].values

numerical_processing = make_pipeline(SimpleImputer(strategy="median"))

categorical_processing = make_pipeline(
SimpleImputer(strategy="constant", fill_value="missing", add_indicator=True),
OneHotEncoder(handle_unknown="ignore"),
)

preprocessing = make_column_transformer(
(numerical_processing, list(np.arange(0, 15))),
(categorical_processing, list(np.arange(15, 17))),
)

n_estimators = args.n_estimators

clf = RandomForestClassifier(n_estimators=n_estimators
,random_state=42)

full_pipeline = make_pipeline(preprocessing, clf)

y = train_data.iloc[:, 0]

full_pipeline.fit(X, y)

joblib.dump(full_pipeline, os.path.join(args.model_dir, 'sklearn_full_pipeline_model.joblib'))

def input_fn(input_data):

return np.array([i for i in input_data.split(",")], dtype="object").reshape(1, -1)

def predict_fn(input_data, model):

return model.predict(input_data)

def model_fn(model_dir):

clf = joblib.load(os.path.join(model_dir, 'sklearn_full_pipeline_model.joblib'))

return clf
```

I am aware that I should pass a custom input_fn to my script so my data input is read correctly, but apparently the default_input_fn is being invoked instead.

Any light on this?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.