aws / aws/amazon-sagemaker-examples

Neo compilation fails for fasterrcnn_resnet50_fpn (torchvision pre-trained model)

Open
#1,501 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

I want to compile fasterrcnn_resnet50_fpn from torchvision pre-trained models using Neo. But in sagemaker console -> compilation job, I got an error
> Failure reason
ClientError: InputConfiguration: Framework cannot load PyTorch model. No module named `'__torch__'`

Step by step to reproduce this issue:

1. I created a new notebook instance (conda_pytorch_p36), and this is some informations about notebook environment:
`torch.__version__ == 1.4.0 `
`torchvision.__version__ == 0.5.0`

2. I loaded fasterrcnn_resnet50_fpn from torchvision and perform scripting:
``` python
model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
trace = torch.jit.script(model.float().eval())
```
3. Then I compared the inference output between original model and scripted model. And I think there's no problem because the output is same.
`image_non_squeze.shape = torch.Size([3, 1680, 2190])`
`image_squeze.shape = torch.Size([1, 3, 1680, 2190])`

``` python
print(model.eval()(image_squeze))
print(trace([image_non_squeze]))
```

> [{'boxes': tensor([[ 265.2566, 232.3313, 2015.6388, 1386.2399],
[ 258.2916, 325.9963, 1114.6427, 1368.3752]], grad_fn=), 'labels': tensor([21, 21]), 'scores': tensor([0.9996, 0.0511], grad_fn=)}]
({}, [{'scores': tensor([0.9996, 0.0511], grad_fn=), 'labels': tensor([21, 21]), 'boxes': tensor([[ 265.2566, 232.3313, 2015.6388, 1386.2399],
[ 258.2916, 325.9963, 1114.6427, 1368.3752]], grad_fn=)}])

4. After that, I saved the traced model `trace.save('model.pth')`, pack it into `model.tar.gz` and upload it to my S3 bucket.
5. Finally I tried to compile traced model using Neo:
`target_device = 'jetson_nano';
framework = 'PYTORCH';
framework_version = '1.2.0'`
``` python
compilation_job_name = 'Faster-RCNN-Neo'

model_key = 'object_detector/model.tar.gz'
model_path = 's3://{}/{}'.format(bucket, model_key)
boto3.resource('s3').Bucket(bucket).upload_file('model.tar.gz', model_key)

compiled_model_path = 's3://{}/{}/output'.format(bucket, compilation_job_name)

data_shape = '{"input0":[3, 800, 800]}'

response = sm_client.create_compilation_job(
CompilationJobName=compilation_job_name,
RoleArn=role,
InputConfig={
'S3Uri': model_path,
'DataInputConfig': data_shape,
'Framework': framework
},
OutputConfig={
'S3OutputLocation': compiled_model_path,
'TargetDevice': target_device
},
StoppingCondition={
'MaxRuntimeInSeconds': 300
}
)
```

This is directory structure of `model.tar.gz` file
![model_structure](https://user-images.githubusercontent.com/31184391/93290848-db8cfc00-f80b-11ea-935d-f759e162cfa4.png)

**[INFO] I have also tried other ways**

1. After I saved the traced model `trace.save()`, pack it into `model.tar.gz` and upload it to my S3 bucket, I created sagemaker pytorch model using `PyTorchModel` class.
``` python
from sagemaker.model import NEO_IMAGE_ACCOUNT
from sagemaker.fw_utils import create_image_uri
from sagemaker.pytorch.model import PyTorchModel
from sagemaker.predictor import RealTimePredictor

image_uri = create_image_uri(region, 'neo-' + framework.lower(), "ml.p3",
framework_version, py_version='py3', account=NEO_IMAGE_ACCOUNT[region])

sagemaker_model = PyTorchModel(model_data=model_path,
image=image_uri,
predictor_cls=RealTimePredictor,
framework_version = framework_version,
role=role,
sagemaker_session=sess,
entry_point='faster_rcnn_neo.py',
py_version='py3'
)
```
2. After that, I used Neo compiler to compile the sagemaker pytorch model
``` python
compiled_model = sagemaker_model.compile(target_instance_family=target_device,
input_shape=data_shape,
job_name=compilation_job_name,
role=role,
framework=framework,
framework_version=framework_version,
output_path=compiled_model_path
)
```
**But the error still same**

> UnexpectedStatusException: Error for Compilation job faster-rcnn-neo-try-1: Failed. Reason: ClientError: InputConfiguration: Framework cannot load PyTorch model. No module named `'__torch__'`

Contributor guide

Open the contributing guide

Research direction

Begin with the SageMaker console compilation job and the boto3 create_compilation_job call, using the supplied model.tar.gz and framework settings. Compare both packaging paths and determine what must change for Neo to load the scripted fasterrcnn_resnet50_fpn model; done when compilation succeeds or the incompatibility is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, jupyter-notebook, python, pytorch
Domain
cloud, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.