aws / aws/sagemaker-scikit-learn-container
Sagemaker sklearn custom code “ValueError: could not convert string to float”
- Dominant language
- Python
- Stars
- 188
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
I am using a sklearn custom script to train and deploy a model in sagemaker. When I tried to invoke the endpoint I had the following error:
`ERROR - model_featurizer_training - Exception on /invocations [POST]
Traceback (most recent call last):
File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper
return fn(*args, **kwargs)
File "/miniconda3/lib/python3.7/site-packages/sagemaker_sklearn_container/serving.py", line 60, in default_input_fn
return np_array.astype(np.float32) if content_type in content_types.UTF8_TYPES else np_array
ValueError: could not convert string to float: 'female'`
My training custom script is as follows:
```
if __name__=='__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--n_estimators', type=int, default=10)
parser.add_argument('--output-data-dir', type=str, default=os.environ['SM_OUTPUT_DATA_DIR'])
parser.add_argument('--model-dir', type=str, default=os.environ['SM_MODEL_DIR'])
parser.add_argument('--train', type=str, default=os.environ['SM_CHANNEL_TRAIN'])
args = parser.parse_args()
input_file = [os.path.join(args.train, file) for file in os.listdir(args.train)]
raw_data = [pd.read_csv(file, engine='python') for file in input_file]
train_data = pd.concat(raw_data)
X = train_data.iloc[:, 1:].values
numerical_processing = make_pipeline(SimpleImputer(strategy="median"))
categorical_processing = make_pipeline(
SimpleImputer(strategy="constant", fill_value="missing", add_indicator=True),
OneHotEncoder(handle_unknown="ignore"),
)
preprocessing = make_column_transformer(
(numerical_processing, list(np.arange(0, 15))),
(categorical_processing, list(np.arange(15, 17))),
)
n_estimators = args.n_estimators
clf = RandomForestClassifier(n_estimators=n_estimators
,random_state=42)
full_pipeline = make_pipeline(preprocessing, clf)
y = train_data.iloc[:, 0]
full_pipeline.fit(X, y)
joblib.dump(full_pipeline, os.path.join(args.model_dir, 'sklearn_full_pipeline_model.joblib'))
def input_fn(input_data):
return np.array([i for i in input_data.split(",")], dtype="object").reshape(1, -1)
def predict_fn(input_data, model):
return model.predict(input_data)
def model_fn(model_dir):
clf = joblib.load(os.path.join(model_dir, 'sklearn_full_pipeline_model.joblib'))
return clf
```
I am aware that I should pass a custom input_fn to my script so my data input is read correctly, but apparently the default_input_fn is being invoked instead.
Any light on this?
Contributor guide
Assessment
This issue has not been assessed yet.