tensorflow / tensorflow/model-optimization

QAT model saving bug: Unable to save function b'__inference_separable_conv2d_layer_call_fn_961'

Open
#964 15 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
1.6k
Forks
349
Avg merge
3d 2h
Merged PRs (30d)
1

Description

Describe the bug
I have a simple model with input layer and a SeparableConv2D layer (note that this issue also happens with Conv2DTranspose). I'm quantizing this model by adding quantize_and_dequantize_v2 nodes at the input and weights of separableconv2d layer (commented in the code). However, I am unable to save the model as SavedModel (whereas converting the Keras model directly to ONNX works):

...
Traceback (most recent call last):
  File "/home/nvidia/PycharmProjects/nvbugs/internal_filed/tf_key_inference_bug/TF_bug_separableconv2d/sample.py", line 24, in <module>
    model.save(model_save_path)
  File "/home/nvidia/PycharmProjects/nvbugs/venv38_trt_regression/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/nvidia/PycharmProjects/nvbugs/venv38_trt_regression/lib/python3.8/site-packages/tensorflow/python/saved_model/save.py", line 403, in map_resources
    raise ValueError(
ValueError: Unable to save function b'__inference_separable_conv2d_layer_call_fn_961' because it captures graph tensor Tensor("model/quant_separable_conv2d/LastValueQuant_1/QuantizeAndDequantizeV4:0", shape=(3, 3, 3, 1), dtype=float32) from a parent function which cannot be converted to a constant with `tf.get_static_value`.

System information

  • TensorFlow version (installed from source or binary): tensorflow-gpu==2.8.0
  • TensorFlow Model Optimization version (installed from source or binary): N/A
  • Python version: 3.8.11

Describe the expected behavior
Keras model can be saved and loaded as SavedModel.

Describe the current behavior
Keras model cannot be saved as SavedModel (loading not able to test since saving is not working).

Code to reproduce the issue
Please download the scripts to reproduce from: https://drive.google.com/file/d/1__EimBaQAXIgNPmYKl99uiBLR8FJe2EN/view?usp=sharing

  1. Requirements:
pip install tensorflow-gpu==2.8.0 tf2onnx
  1. Reproduce the issue by running:
python sample_qat.py

Additional context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by downloading the linked reproduction scripts and running sample_qat.py with TensorFlow 2.8.0. Inspect the quantize_and_dequantize_v2 usage around SeparableConv2D and Conv2DTranspose, then verify that the Keras model can be saved and loaded as a SavedModel without the captured graph tensor error.

Written by the indexing model from the issue text.

Assessment

Tech stack
keras, python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.