ERROR: maximum recursion depth exceeded when using linear_quantize_activations
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## 🐞Describing the bug
I get a `maximum recursion depth exceeded` error when trying to quantize a transformer model using `coreml.experimental.linear_quantize_activations`. When I increase the recursion limit, I get
```killed python /Users/tadghk/coreml-onnx-export/coreml_inference.py
/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
```
My system is an M1 Macbook, and I'm using `coremltoools 8.0`. Unfortunately, I cannot share the model.
## Stack Trace
```
(direct-core) tadghk@Tadghs-MacBook-Air coreml-onnx-export % python /Users/tadghk/coreml-onnx-export/coreml_inference.py
Running activation compression pass insert_prefix_quantize_dequantize_pair: 100%|█| 4053/4053 [00:0
Running compression pass linear_quantize_activations: start calibrating 1 samples
Running compression pass linear_quantize_activations: calibration may take a while ...
Traceback (most recent call last):
File "/Users/tadghk/coreml-onnx-export/coreml_inference.py", line 16, in
compressed_model_a8 = cto.coreml.experimental.linear_quantize_activations(
File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_post_training_quantization.py", line 128, in linear_quantize_activations
activation_stats = _get_activation_calibration_stats(mlmodel, sample_data)
File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_post_training_quantization.py", line 285, in _get_activation_calibration_stats
debugger = ModelDebugger(fpmodel)
File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 187, in __init__
all_outputs = self.__class__.get_all_outputs(self.block_info)
File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 139, in get_all_outputs
cls.populate_outputs(output_names, block_info.operations, acc)
File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 133, in populate_outputs
cls.populate_outputs(prev_output_names, all_operations, acc)
File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 133, in populate_outputs
cls.populate_outputs(prev_output_names, all_operations, acc)
File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 133, in populate_outputs
cls.populate_outputs(prev_output_names, all_operations, acc)
[Previous line repeated 988 more times]
File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 132, in populate_outputs
prev_output_names = cls.unique(prev_output_names)
File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 58, in unique
return [x for x in sequence if not (x in seen or seen.add(x))]
File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 58, in
return [x for x in sequence if not (x in seen or seen.add(x))]
File "/opt/homebrew/anaconda3/envs/direct-core/lib/python3.10/site-packages/coremltools/optimize/coreml/experimental/_model_debugger.py", line 58, in
return [x for x in sequence if not (x in seen or seen.add(x))]
RecursionError: maximum recursion depth exceeded while calling a Python object
```
## To Reproduce
```
import coremltools as ct
import coremltools.optimize as cto
import numpy as np
import time
import sys
sys.setrecursionlimit(50000)
model = ct.models.MLModel('./coreml_model.mlpackage',
# optimization_hints = {'specializationStrategy': ct.SpecializationStrategy.FastPrediction}
)
activation_config = cto.coreml.OptimizationConfig(
global_config=cto.coreml.experimental.OpActivationLinearQuantizerConfig(
mode="linear_symmetric"
)
)
compressed_model_a8 = cto.coreml.experimental.linear_quantize_activations(
model, activation_config, [{'img':np.random.randn(1,13,1024,1024)}]
)
```
# Paste Python code snippet here, complete with any required import statements.
```
- If the model conversion succeeds, but there is a numerical mismatch in predictions, please include the code used for comparisons.
## System environment (please complete the following information):
- coremltools version: 8.0
- OS (e.g. MacOS version or Linux type): MacOS
Contributor guide
Research direction
Start with coremltools/optimize/coreml/experimental/_model_debugger.py, especially ModelDebugger.populate_outputs, and trace how linear_quantize_activations reaches it through _post_training_quantization.py. Run the supplied reproduction if the Core ML model is available; done means activation quantization no longer fails with RecursionError on the reported deep model.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100