apple / apple/coremltools

Auto-Fuse ReLU with BatchNorm

Open
#670 4 comments 0 reactions 0 assignees View on GitHub
perf question
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

When constructing my model in `tf.keras`, I have a repeated sequence of layers that goes:

```
tf.keras.layers.Conv2D(...)
tf.keras.layers.BatchNormalization(...)
tf.keras.layers.ReLU(...)
```

Due to the BatchNorm layer in between the convolution and activation function, I cannot specify a` Fused ReLU `directly within the `Conv2D` layer. Therefore, I lose out on reaping the benefits of `Fused ReLU`.

When converting the model to CoreML, `BatchNorm` layers are automatically fused in so we are just left with: `Conv2D -> ReLU` after model conversion.

I would like to see coremltools **automatically** fuse ReLU into the `Fused Conv2D+BN` layer (currently, when inspecting the model with Netron, I see that "Convolution" and "activation' are separate layers).

This would be pretty convenient for a lot of ML practitioners looking to optimize their models.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the CoreML conversion of the tf.keras Conv2D-BatchNormalization-ReLU sequence and compare the converted model in Netron. Done means the converted model represents the convolution, batch normalization, and ReLU as a fused layer rather than separate Convolution and activation layers.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.