tensorflow / tensorflow/model-optimization

Fail to fuse ReLU6 after QAT by using keras.layers.Activation('relu6')

Open
#1,030 1 comment 0 reactions 1 assignee View on GitHub

@cdh4696 is already working on this.

Since Jan 11, 2023.

bug
Dominant language
Python
Stars
1.6k
Forks
349
Avg merge
3d 2h
Merged PRs (30d)
1

Description

Describe the bug
ReLU6 cannot be fused into Conv2D (Conv2D+BN+ReLU6) after QAT by using keras.layers.Activation('relu6')

Following works fine:

  • keras.layers.ReLU(6)
  • keras.layers.Activation('relu')
  • keras.layers.ReLU()

So the workaround is to stick to keras.layers.ReLU(6)

System information

TensorFlow version (installed from source or binary): 2.9.2 (colab default)

TensorFlow Model Optimization version (installed from source or binary): 0.7.3 (pip default)

Python version: 3.8 (colab default)

Describe the expected behavior
keras.layers.Activation('relu6') should be fused into Conv2D as same as keras.layers.Activation('relu')

Describe the current behavior
keras.layers.Activation('relu6') failed to be fused into Conv2D

Code to reproduce the issue
Colab: https://colab.research.google.com/drive/1tuGvsuBsUiWUdks_i9glXgdUoUcFSXqI

Screenshots
keras.layers.Activation('relu6')
圖片

keras.layers.ReLU(6)
圖片

Additional context
It is convenient for users to configure the model's activation by keras.layers.Activation(...). It might take some time to detect the strange behavior by using keras.layers.Activation('relu6').

Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.