apple / apple/coremltools

Error while running 8_bit_quantize_weights method

Open
#1,937 8 comments 0 reactions 1 assignee Claimed by @YifanShenSZ View on GitHub
bug triaged
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describing the bug
With successfully converted model from Tenserflow to CoreML format, I wanted try to quantize model. I run snippet below with model in `.mlpackage` format but got an error.

## Stack Trace
---------------------------------------------------------------------------
```
ValueError Traceback (most recent call last)
Cell In[691], line 20
14 import coremltools.optimize as cto
16 config = cto.coreml.OptimizationConfig(
17 global_config=cto.coreml.OpLinearQuantizerConfig(mode="linear_symmetric")
18 )
---> 20 compressed_8_bit_model = cto.coreml.linear_quantize_weights(mlModel, config)
21 compressed_8_bit_model.save("[./RockPaperScissors_8bit.mlpackage](https://file+.vscode-resource.vscode-cdn.net/Users/stepanzalis/Desktop/Rock-paper-scissors/RockPaperScissors_8bit.mlpackage)")

File [/opt/homebrew/lib/python3.10/site-packages/coremltools/optimize/coreml/_post_training_quantization.py:172](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.10/site-packages/coremltools/optimize/coreml/_post_training_quantization.py:172), in linear_quantize_weights(mlmodel, config)
62 """
63 Utility function to convert a float precision MLModel of type ``mlprogram``, which uses
64 float-precision weights, into a compressed MLModel that uses 8-bit weights. This is
(...)
168
169 """
171 linear_weight_quantizer = _linear_quantize_weights(config, fake_compression=False)
--> 172 return _apply_graph_pass(mlmodel, linear_weight_quantizer)
File [/opt/homebrew/lib/python3.10/site-packages/coremltools/optimize/coreml/_post_training_quantization.py:51](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.10/site-packages/coremltools/optimize/coreml/_post_training_quantization.py:51), in _apply_graph_pass(mlmodel, graph_pass)
48 graph_pass.apply(prog)
50 # convert the pymil program back to mlmodel
---> 51 compressed_mlmodel = _mil_convert(
52 prog,
...
289 @property
290 def shape(self):
--> 291 raise ValueError("shape not applicable to ListVar '{}'.".format(self.name))
ValueError: shape not applicable to ListVar 'const_13'.
```
---------------------------------------------------------------------------

## To Reproduce
Repreduced by snippet with model attached below.

```
import coremltools as ct
import coremltools.optimize as cto

model = ct.models.MLModel("./RockPaperScissors.mlpackage")

config = cto.coreml.OptimizationConfig(
global_config=cto.coreml.OpLinearQuantizerConfig(mode="linear_symmetric")
)

compressed_8_bit_model = cto.coreml.linear_quantize_weights(mlModel, config)
compressed_8_bit_model.save("./RockPaperScissors_8bit.mlpackage")
```

## System environment (please complete the following information):
- coremltools version: 7.0b1
- OS: MacOS 13.4.1
- Tenserflow 2.13.0

## Additional info
- Model predictions are the same as the original Tenserflow model.
- Model can be downloaded here: https://drive.google.com/file/d/1X-1d9lMlrjSYnqW3fL9arziwWoU0qkac/view?usp=sharing
- I noted none of the optimisation methods works. Could it be related to architecture of the model? It's basically MobileNetV2 with one extra Dropout layer and output layer for classification (only 3 outputs).
- When converting to .mlmodel, it works with older API.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.