aws / aws/amazon-sagemaker-examples
AutoGluon model endpoint creation using Batch Transform
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
The endpoint shall run only when its required as this will save cost by invoking the machine when required.
error -
2023-03-29T13:16:50.587:[sagemaker logs]: MaxConcurrentTransforms=1, MaxPayloadInMB=6, BatchStrategy=MULTI_RECORD
2023-03-29T13:16:50.813:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: Bad HTTP status received from algorithm: 500
2023-03-29T13:16:50.813:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv:
2023-03-29T13:16:50.813:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: Message:
2023-03-29T13:16:50.813:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: could not convert string to float: 'F'
2023-03-29T13:16:50.814:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: Traceback (most recent call last):
2023-03-29T13:16:50.814:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: File "/usr/local/lib/python3.8/dist-packages/sagemaker_inference/transformer.py", line 128, in transform
2023-03-29T13:16:50.814:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: result = self._transform_fn(self._model, input_data, content_type, accept)
2023-03-29T13:16:50.814:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: File "/opt/ml/model/code/tabular_serve-batch.py", line 32, in transform_fn
2023-03-29T13:16:50.814:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: pred = model.predict(data)
2023-03-29T13:16:50.814:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: File "/usr/local/lib/python3.8/dist-packages/autogluon/tabular/predictor/predictor.py", line 1382, in predict
2023-03-29T13:16:50.814:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: return self._learner.predict(X=data, model=model, as_pandas=as_pandas, transform_features=transform_features)
2023-03-29T13:16:50.814:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: File "/usr/local/lib/python3.8/dist-packages/autogluon/tabular/learner/abstract_learner.py", line 153, in predict
2023-03-29T13:16:50.814:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: y_pred_proba = self.predict_proba(X=X, model=model, as_pandas=False, as_multiclass=False, inverse_transform=False, transform_features=transform_features)
2023-03-29T13:16:50.814:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: File "/usr/local/lib/python3.8/dist-packages/autogluon/tabular/learner/abstract_learner.py", line 133, in predict_proba
2023-03-29T13:16:50.814:[sagemaker logs]: s3-ampf-us-east-1-aim-gco-expl-c3/SageMaker_Endpoint_Output_Test/test.csv: X = self.transform_features(X)
---------------------------------------------------------------------------
UnexpectedStatusException Traceback (most recent call last)
in
/opt/conda/lib/python3.7/site-packages/sagemaker/workflow/pipeline_context.py in wrapper(*args, **kwargs)
270 return _StepArguments(retrieve_caller_name(self_instance), run_func, *args, **kwargs)
271
--> 272 return run_func(*args, **kwargs)
273
274 return wrapper
/opt/conda/lib/python3.7/site-packages/sagemaker/transformer.py in transform(self, data, data_type, content_type, compression_type, split_type, job_name, input_filter, output_filter, join_source, experiment_config, model_client_config, batch_data_capture_config, wait, logs)
273
274 if wait:
--> 275 self.latest_transform_job.wait(logs=logs)
276
277 def transform_with_monitoring(
/opt/conda/lib/python3.7/site-packages/sagemaker/transformer.py in wait(self, logs)
623 def wait(self, logs=True):
624 if logs:
--> 625 self.sagemaker_session.logs_for_transform_job(self.job_name, wait=True)
626 else:
627 self.sagemaker_session.wait_for_transform_job(self.job_name)
/opt/conda/lib/python3.7/site-packages/sagemaker/session.py in logs_for_transform_job(self, job_name, wait, poll)
4384
4385 if wait:
-> 4386 self._check_job_status(job_name, description, "TransformJobStatus")
4387 if dot:
4388 print()
/opt/conda/lib/python3.7/site-packages/sagemaker/session.py in _check_job_status(self, job, desc, status_key_name)
3750 message=message,
3751 allowed_statuses=["Completed", "Stopped"],
-> 3752 actual_status=status,
3753 )
3754
UnexpectedStatusException: Error for Transform job autogluon-inference-2023-03-29-13-10-47-371: Failed. Reason: AlgorithmError: See job logs for more information
Contributor guide
Assessment
This issue has not been assessed yet.