apple / apple/coremltools

[Torch] Support FP16 Conversion

Open
#2,429 0 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

As of now, our torch converter
1. Assumes the given torch model is in fp32 compute precision (i.e. weights and activations are all in fp32)
2. Converts torch model as is (i.e. no treatments such as promoting types)
3. Sandwichs ops with `cast(fp16) -> op -> cast(fp32)` then eliminate cancelling casts to obtain fp16 compute precision

This works in most cases: people usually can call `torch_model.to(torch.float32)` then invoke `coremltools.convert`. However, there are cases where developers request conversion support for fp16 or mixed fp16-fp32 torch models
1. https://github.com/apple/coremltools/pull/2423 fp32 torch model would be too big to fit in memory
2. https://github.com/apple/coremltools/pull/2274
3. https://github.com/apple/coremltools/pull/2241

Contributor guide

Open the contributing guide

Research direction

Start by locating the torch converter and review the referenced pull requests 2423, 2274, and 2241 for the fp16 and mixed fp16-fp32 cases they describe. Done should mean that torch models using fp16 or mixed precision can be converted without requiring conversion from an fp32 model.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.