aws / aws/amazon-sagemaker-examples
Neo compilation fails for fasterrcnn_resnet50_fpn (torchvision pre-trained model)
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
I want to compile fasterrcnn_resnet50_fpn from torchvision pre-trained models using Neo. But in sagemaker console -> compilation job, I got an error
> Failure reason
ClientError: InputConfiguration: Framework cannot load PyTorch model. No module named `'__torch__'`
Step by step to reproduce this issue:
1. I created a new notebook instance (conda_pytorch_p36), and this is some informations about notebook environment:
`torch.__version__ == 1.4.0 `
`torchvision.__version__ == 0.5.0`
2. I loaded fasterrcnn_resnet50_fpn from torchvision and perform scripting:
``` python
model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
trace = torch.jit.script(model.float().eval())
```
3. Then I compared the inference output between original model and scripted model. And I think there's no problem because the output is same.
`image_non_squeze.shape = torch.Size([3, 1680, 2190])`
`image_squeze.shape = torch.Size([1, 3, 1680, 2190])`
``` python
print(model.eval()(image_squeze))
print(trace([image_non_squeze]))
```
> [{'boxes': tensor([[ 265.2566, 232.3313, 2015.6388, 1386.2399],
[ 258.2916, 325.9963, 1114.6427, 1368.3752]], grad_fn=), 'labels': tensor([21, 21]), 'scores': tensor([0.9996, 0.0511], grad_fn=)}]
({}, [{'scores': tensor([0.9996, 0.0511], grad_fn=), 'labels': tensor([21, 21]), 'boxes': tensor([[ 265.2566, 232.3313, 2015.6388, 1386.2399],
[ 258.2916, 325.9963, 1114.6427, 1368.3752]], grad_fn=)}])
4. After that, I saved the traced model `trace.save('model.pth')`, pack it into `model.tar.gz` and upload it to my S3 bucket.
5. Finally I tried to compile traced model using Neo:
`target_device = 'jetson_nano';
framework = 'PYTORCH';
framework_version = '1.2.0'`
``` python
compilation_job_name = 'Faster-RCNN-Neo'
model_key = 'object_detector/model.tar.gz'
model_path = 's3://{}/{}'.format(bucket, model_key)
boto3.resource('s3').Bucket(bucket).upload_file('model.tar.gz', model_key)
compiled_model_path = 's3://{}/{}/output'.format(bucket, compilation_job_name)
data_shape = '{"input0":[3, 800, 800]}'
response = sm_client.create_compilation_job(
CompilationJobName=compilation_job_name,
RoleArn=role,
InputConfig={
'S3Uri': model_path,
'DataInputConfig': data_shape,
'Framework': framework
},
OutputConfig={
'S3OutputLocation': compiled_model_path,
'TargetDevice': target_device
},
StoppingCondition={
'MaxRuntimeInSeconds': 300
}
)
```
This is directory structure of `model.tar.gz` file

**[INFO] I have also tried other ways**
1. After I saved the traced model `trace.save()`, pack it into `model.tar.gz` and upload it to my S3 bucket, I created sagemaker pytorch model using `PyTorchModel` class.
``` python
from sagemaker.model import NEO_IMAGE_ACCOUNT
from sagemaker.fw_utils import create_image_uri
from sagemaker.pytorch.model import PyTorchModel
from sagemaker.predictor import RealTimePredictor
image_uri = create_image_uri(region, 'neo-' + framework.lower(), "ml.p3",
framework_version, py_version='py3', account=NEO_IMAGE_ACCOUNT[region])
sagemaker_model = PyTorchModel(model_data=model_path,
image=image_uri,
predictor_cls=RealTimePredictor,
framework_version = framework_version,
role=role,
sagemaker_session=sess,
entry_point='faster_rcnn_neo.py',
py_version='py3'
)
```
2. After that, I used Neo compiler to compile the sagemaker pytorch model
``` python
compiled_model = sagemaker_model.compile(target_instance_family=target_device,
input_shape=data_shape,
job_name=compilation_job_name,
role=role,
framework=framework,
framework_version=framework_version,
output_path=compiled_model_path
)
```
**But the error still same**
> UnexpectedStatusException: Error for Compilation job faster-rcnn-neo-try-1: Failed. Reason: ClientError: InputConfiguration: Framework cannot load PyTorch model. No module named `'__torch__'`
Contributor guide
Research direction
Begin with the SageMaker console compilation job and the boto3 create_compilation_job call, using the supplied model.tar.gz and framework settings. Compare both packaging paths and determine what must change for Neo to load the scripted fasterrcnn_resnet50_fpn model; done when compilation succeeds or the incompatibility is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, jupyter-notebook, python, pytorch
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100