facebookresearch / facebookresearch/detectron2

Caffe2 export of quantized model

Open
#1,496 24 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

Hello, everyone

First of all I've finally obtained results in a fight with ONNX export of quantized model built on detectron2 and trained using PyTorch QAT tools. So if anyone interested in guidance in this or in PR of additional functionality in detectron2 please write me back.

I understand that export of quantized model using `detectron2/export` tools might not be supported for now. But there are frequent mentions of INT8 in a code that's why I'll ask the following.

While I do conversion after I see:
```ONNX export Done. Exported predict_net (before optimizations)```

Code logs an error:
```
[W init.h:137] Caffe2 GlobalInit should be run before any other API calls.
[W tensor.cc:167] Changing the data type of Tensor is discouraged. Attempt to change data type from: nullptr (uninitialized) to: unsigned char
Traceback (most recent call last):
File "./tools/caffe2_converter.py", line 111, in
caffe2_model = export_caffe2_model(cfg, model, first_batch)
File "/root/some_detectron2/detectron2/export/api.py", line 157, in export_caffe2_model
return Caffe2Tracer(cfg, model, inputs).export_caffe2()
File "/root/some_detectron2/detectron2/export/api.py", line 95, in export_caffe2
predict_net, init_net = export_caffe2_detection_model(model, inputs)
File "/root/some_detectron2/detectron2/export/caffe2_export.py", line 171, in export_caffe2_detection_model
init_net = construct_init_net_from_params(params, device_options)
File "/root/some_detectron2/detectron2/export/shared.py", line 312, in construct_init_net_from_params
[create_const_fill_op(name, blob, device_option=device_options.get(name, None))]
File "/root/some_detectron2/detectron2/export/shared.py", line 290, in create_const_fill_op
assert device_option is None
AssertionError
```

The model takes floats as input and returns floats as outputs, inputs are implicitly quantized at the beggining of the network.

Is it possible to properly configurate export API functionality to export qunatized model?
Is `device_option` assertion is vital in such exporting?

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with tools/caffe2_converter.py and read detectron2/export/api.py, caffe2_export.py, and shared.py around construct_init_net_from_params and create_const_fill_op. Done means the export API can properly export the quantized model described in the issue without the device_option assertion.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.