tensorflow / tensorflow/model-optimization

Symmetric quantization returns nonzero value for 0 on GPU

Open
#635 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
1.6k
Forks
349
Avg merge
3d 2h
Merged PRs (30d)
1

Description

Describe the bug
In symmetric quantization, after (fake) quantization, 0 should be exactly 0.
But model-optimization uses the fake quantization API designed for asymmetric quantization so that it could return nonzero values under certain conditions.

System information
Ubuntu 16.04
Tesla V100-DGXS-32GB
Tested in container tensorflow/tensorflow:nightly-gpu

TensorFlow version (installed from source or binary):
nightly container tensorflow/tensorflow:nightly-gpu

TensorFlow Model Optimization version (installed from source or binary):
master

Python version:
3.6.9

Describe the expected behavior
0 returns 0 after symmetric quantization

Describe the current behavior
0 returns a tiny floating point number after symmetric quantization

Code to reproduce the issue

import tensorflow as tf
x = tf.constant([11., 9., 0., 12])
with tf.device("/GPU:0"):
  fq_x = tf.quantization.fake_quant_with_min_max_args(x, -10., 10.)
  fq_x_narrow = tf.quantization.fake_quant_with_min_max_args(x, -10., 10., narrow_range=True)
  qdq_x = tf.quantization.quantize_and_dequantize_v2(x, -10., 10., range_given=True)
print(fq_x.numpy())
print(fq_x_narrow.numpy())
print(qdq_x.numpy())
with tf.device("/CPU:0"):
  fq_x = tf.quantization.fake_quant_with_min_max_args(x, -10., 10.)
  fq_x_narrow = tf.quantization.fake_quant_with_min_max_args(x, -10., 10., narrow_range=True)
  qdq_x = tf.quantization.quantize_and_dequantize_v2(x, -10., 10., range_given=True)
print(fq_x.numpy())
print(fq_x_narrow.numpy())
print(qdq_x.numpy())

Result

[ 1.0039216e+01  9.0196075e+00 -2.4586916e-07  1.0039216e+01]
[ 1.0000000e+01  8.9763775e+00 -3.7252903e-08  1.0000000e+01]
[10.         8.9763775  0.        10.       ]
[10.039215  9.019608  0.       10.039215]
[10.         8.9763775  0.        10.       ]
[10.         8.9763775  0.        10.       ]

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
The problem is model-optimization uses tf.quantization.fake_quant_with_min_max_vars for "symmetric" quantization https://github.com/tensorflow/model-optimization/blob/master/tensorflow_model_optimization/python/core/quantization/keras/quant_ops.py#L87-L113.
fake_quant_with_min_max_vars are not for symmetric quantization. The one designed for symmetric fake quantization is https://www.tensorflow.org/api_docs/python/tf/quantization/quantize_and_dequantize.

The problem seems to affect GPU only. CPU path returns exactly 0.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tensorflow_model_optimization/python/core/quantization/keras/quant_ops.py at the symmetric quantization path around lines 87-113, then compare its fake-quantization API with the quantize_and_dequantize behavior shown in the reproduction. Run the GPU and CPU examples to confirm the discrepancy; done means symmetric quantization returns exactly 0 for an input of 0 on GPU.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.