tensorflow / tensorflow/model-optimization

Post training quantize TTS model fastspeech2 and failed

Open
#1,010 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
1.6k
Forks
349
Avg merge
3d 2h
Merged PRs (30d)
1

Description

Hi Everyone,

I tried to post quantize fastspeech2 model of TensorFlowTTS ,but got the error message, I was wondering if anyone knows the detials of this error and how to fix this ? Thanks!

Use python version : 3.8.10
Use tensorflow version : 2.8.0

test code:

import tensorflow as tf
import yaml
import numpy as np
from tensorflow_tts.configs import FastSpeech2Config
from tensorflow_tts.models import TFFastSpeech2

  def representative_data_gen():

       data = np.array([38,51,41,11,51,52,57,11,57,2,11,45,46,56,11,55,38,40,42,7,148])  
       data = data.reshape(1,-1)
       input_ids = tf.convert_to_tensor(data, dtype=tf.int32)
            
        yield [input_ids,
         tf.convert_to_tensor([0], tf.int32),
         tf.convert_to_tensor([1.0], dtype=tf.float32),
         tf.convert_to_tensor([1.0], dtype=tf.float32),
         tf.convert_to_tensor([1.0], dtype=tf.float32)]
  
  with open('config.yml') as f:
      config = yaml.load(f, Loader=yaml.Loader)
  
  config = FastSpeech2Config(**config["fastspeech2_params"])
  fastspeech2 = TFFastSpeech2(config=config, enable_tflite_convertible=True, name="fastspeech2")
  fastspeech2._build()
  fastspeech2.load_weights("model.h5")
  
  fastspeech2_concrete_function = fastspeech2.inference_tflite.get_concrete_function()
  converter = tf.lite.TFLiteConverter.from_concrete_functions([fastspeech2_concrete_function])
  converter.optimizations = [tf.lite.Optimize.DEFAULT]
  converter.representative_dataset = representative_data_gen
  converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
  converter.inference_input_type = tf.compat.v1.lite.constants.INT8
  converter.inference_output_type = tf.compat.v1.lite.constants.INT8
  
  tflite_model_quant = converter.convert()

error log:

2022-08-23 09:18:33.257335: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:1164] Optimization results for grappler item: graph_to_optimize
   function_optimizer: function_optimizer did nothing. time = 0.011ms.
   function_optimizer: function_optimizer did nothing. time = 0.001ms.
 
 /usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/convert.py:746: UserWarning: Statistics for quantized inputs were expected, but not specified; continuing anyway.
   warnings.warn("Statistics for quantized inputs were expected, but not "
 2022-08-23 09:18:35.605349: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:357] Ignored output_format.
 2022-08-23 09:18:35.605393: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:360] Ignored drop_control_dependency.
 fully_quantize: 0, inference_type: 6, input_inference_type: 9, output_inference_type: 9
 error: illegal scale: INF
 Segmentation fault (core dumped)

Models information and weights are from here:
https://huggingface.co/tensorspeech/tts-fastspeech2-ljspeech-en/tree/main
https://github.com/TensorSpeech/TensorFlowTTS

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the supplied FastSpeech2 conversion with Python 3.8.10, TensorFlow 2.8.0, the linked model weights, and the representative_data_gen function. Inspect the TFLite conversion path around the “illegal scale: INF” error and determine whether the issue is reproducible; done means a confirmed cause with a documented fix or workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.