[Torch] Support FP16 Conversion
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
As of now, our torch converter
1. Assumes the given torch model is in fp32 compute precision (i.e. weights and activations are all in fp32)
2. Converts torch model as is (i.e. no treatments such as promoting types)
3. Sandwichs ops with `cast(fp16) -> op -> cast(fp32)` then eliminate cancelling casts to obtain fp16 compute precision
This works in most cases: people usually can call `torch_model.to(torch.float32)` then invoke `coremltools.convert`. However, there are cases where developers request conversion support for fp16 or mixed fp16-fp32 torch models
1. https://github.com/apple/coremltools/pull/2423 fp32 torch model would be too big to fit in memory
2. https://github.com/apple/coremltools/pull/2274
3. https://github.com/apple/coremltools/pull/2241
Contributor guide
Research direction
Start by locating the torch converter and review the referenced pull requests 2423, 2274, and 2241 for the fp16 and mixed fp16-fp32 cases they describe. Done should mean that torch models using fp16 or mixed precision can be converted without requiring conversion from an fp32 model.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100