aws / aws/amazon-sagemaker-examples

Failed to publish an endpoint with custom sklearn model

Open
#1,783 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

Hi,

I created a custom model from MultiOutputRegressor of sklearn.
The train worked but I have the error AttributeError: module 'main' has no attribute 'DataTransformer' when publishing an end point of the model.
- script.py is my script to train model
- DataTransformer.py is the file where I declare a custom class from the MultiOutputRegressor of sklearn.

I found a topic with the same issue here and tried the proposition but it's still not working. [https://github.com/aws/amazon-sagemaker-examples/issues/725]

Here's my settings:

estimator = SKLearn(
entry_point="script.py",
role=role_name,
train_instance_count=1, # training instance count
train_instance_type=instance_type, # training instance type
output_path=f's3://{bucket}/{prefix}/output', # S3 location for output data
sagemaker_session=sess,
framework_version='0.23-1',
base_job_name=base_job_name,
hyperparameters={'data_path': dataset_to_train},
dependencies=['DataTransformer.py'],
source_dir='s3://mybucket/pyscripts/source.tar.gz')

The files script.py and DataTransformer.py is zipped and uploaded on S3, the 'source_dir' points to the .tar.gz file.
How would I modify my script to make it work?

Contributor guide

Open the contributing guide

Research direction

Start by reviewing script.py, DataTransformer.py, and the SKLearn estimator's dependencies and source_dir settings, then reproduce endpoint publication with the packaged archive. Trace why the custom DataTransformer class is looked up in main and verify that publishing completes without the AttributeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python, scikit-learn
Domain
cloud, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.