tensorflow / tensorflow/model-optimization
QAT model saving bug: Unable to save function b'__inference_separable_conv2d_layer_call_fn_961'
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 349
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 1
Description
Describe the bug
I have a simple model with input layer and a SeparableConv2D layer (note that this issue also happens with Conv2DTranspose). I'm quantizing this model by adding quantize_and_dequantize_v2 nodes at the input and weights of separableconv2d layer (commented in the code). However, I am unable to save the model as SavedModel (whereas converting the Keras model directly to ONNX works):
...
Traceback (most recent call last):
File "/home/nvidia/PycharmProjects/nvbugs/internal_filed/tf_key_inference_bug/TF_bug_separableconv2d/sample.py", line 24, in <module>
model.save(model_save_path)
File "/home/nvidia/PycharmProjects/nvbugs/venv38_trt_regression/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/nvidia/PycharmProjects/nvbugs/venv38_trt_regression/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py", line 403, in map_resources
raise ValueError(
ValueError: Unable to save function b'__inference_separable_conv2d_layer_call_fn_961' because it captures graph tensor Tensor("model/quant_separable_conv2d/LastValueQuant_1/QuantizeAndDequantizeV4:0", shape=(3, 3, 3, 1), dtype=float32) from a parent function which cannot be converted to a constant with `tf.get_static_value`.
System information
- TensorFlow version (installed from source or binary):
tensorflow-gpu==2.8.0 - TensorFlow Model Optimization version (installed from source or binary): N/A
- Python version: 3.8.11
Describe the expected behavior
Keras model can be saved and loaded as SavedModel.
Describe the current behavior
Keras model cannot be saved as SavedModel (loading not able to test since saving is not working).
Code to reproduce the issue
Please download the scripts to reproduce from: https://drive.google.com/file/d/1__EimBaQAXIgNPmYKl99uiBLR8FJe2EN/view?usp=sharing
- Requirements:
pip install tensorflow-gpu==2.8.0 tf2onnx
- Reproduce the issue by running:
python sample_qat.py
Additional context
- This issue also happened with
DepthwiseConv2Dintf<2.8.0, but has been fixed in 2.8.0. Please see the previous issue here: https://github.com/tensorflow/model-optimization/issues/868 . - This issue also happens with
ConvTranspose2D.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by downloading the linked reproduction scripts and running sample_qat.py with TensorFlow 2.8.0. Inspect the quantize_and_dequantize_v2 usage around SeparableConv2D and Conv2DTranspose, then verify that the Keras model can be saved and loaded as a SavedModel without the captured graph tensor error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- keras, python, tensorflow
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100