ct.convert(convert_to="mlprogram", compute_precision=ct.precision.FLOAT16) have precision problem for some models
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## 🐞Describing the bug
- I have checked my cases, and found for some models, ct.convert(convert_to="mlprogram", compute_precision=ct.precision.FLOAT16) have big problem of precision, while ct.convert(convert_to="mlprogram", compute_precision=ct.precision.FLOAT32) is correct. also ct.convert(convert_to="neuralnetwork") along with quantization_utils.quantize_weights(model_ct, nbits=16) is correct.
## Stack Trace
-
## To Reproduce
- sorry cannot provide model now
## System environment (please complete the following information):
- coremltools version: 8.3.0
- OS (e.g. MacOS version or Linux type): MacOS_14.5 (23F79)
- Any other relevant version information (e.g. PyTorch or TensorFlow version): pytorch_2.2.2
## Additional context
- https://github.com/apple/coremltools/issues/2301
- https://github.com/apple/coremltools/issues/2481
- and so on
Contributor guide
Research direction
Start at the ct.convert entry point and compare the FLOAT16 and FLOAT32 mlprogram paths with the neuralnetwork plus quantize_weights path described in the report. A reproducible model is needed before the precision difference can be investigated; done means identifying the conversion cause and confirming corrected precision with a regression case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100