tensorflow / tensorflow/model-optimization
wrong inference results of qat int8 model with Conv2DTranspose.
Open
@Xhark is already working on this.
Since Aug 4, 2021.
bug
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 349
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 1
Description
Tensorflow == 2.4.0
Below is my image compression model which have two Conv2DTranspose layers.
model = keras.Sequential([
keras.layers.InputLayer(input_shape=(256, 256, 3)),
keras.layers.Conv2D(filters=64, kernel_size=(4, 4), strides=(2, 2), padding='same',activation=None),
keras.layers.Conv2D(filters=96, kernel_size=(4, 4), strides=(2, 2), padding='same',activation=None),
keras.layers.Conv2DTranspose(filters=128, kernel_size=(4, 4), strides=(2, 2), padding='same',activation=None),
keras.layers.Conv2DTranspose(filters=3, kernel_size=(4, 4), strides=(2, 2), padding='same', activation=None)
])
After quant aware train the model, calculate MSE between inputs and outputs, inference result of tflite is totally wrong.
inference Keras with fake quant op: MSE is 1.6923981
inference tflite : MSE is 4376.958.
After debug, the error is caused by Conv2DTranspose. Just inference the last deconv layer,results as below.
>>> inference_tflite.reshape(-1)[:10]
array([-0.6456693 , 0. , 0. , -0.68503934, 0. ,
0. , -0.70866144, 0. , 0. , -0.70866144],
dtype=float32)
>>> inference_keras.reshape(-1)[:10]
array([-0.6456693 , -0.5984252 , -0.53543305, -0.6850394 , -0.62204725,
-0.53543305, -0.70866144, -0.62992126, -0.5511811 , -0.70866144],
dtype=float32)
Compared results, only the first channel of tflite result is correct,the else are all zeros. Why?
tflite TestInputs
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.