tensorflow / tensorflow/model-optimization
Fail to fuse ReLU6 after QAT by using keras.layers.Activation('relu6')
@cdh4696 is already working on this.
Since Jan 11, 2023.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 349
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 1
Description
Describe the bug
ReLU6 cannot be fused into Conv2D (Conv2D+BN+ReLU6) after QAT by using keras.layers.Activation('relu6')
Following works fine:
keras.layers.ReLU(6)keras.layers.Activation('relu')keras.layers.ReLU()
So the workaround is to stick to keras.layers.ReLU(6)
System information
TensorFlow version (installed from source or binary): 2.9.2 (colab default)
TensorFlow Model Optimization version (installed from source or binary): 0.7.3 (pip default)
Python version: 3.8 (colab default)
Describe the expected behavior
keras.layers.Activation('relu6') should be fused into Conv2D as same as keras.layers.Activation('relu')
Describe the current behavior
keras.layers.Activation('relu6') failed to be fused into Conv2D
Code to reproduce the issue
Colab: https://colab.research.google.com/drive/1tuGvsuBsUiWUdks_i9glXgdUoUcFSXqI
Screenshots
keras.layers.Activation('relu6')

keras.layers.ReLU(6)

Additional context
It is convenient for users to configure the model's activation by keras.layers.Activation(...). It might take some time to detect the strange behavior by using keras.layers.Activation('relu6').
Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.