TensorflowTTS Keras model fails to export.
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## πDescribing the bug
Attempting to convert some of the text to speech models at https://github.com/TensorSpeech/TensorFlowTTS
The models work correctly under tensorflowlite/coreml but fail to directly export using CoreML Tools. I've included the stack trace and example code. You will need to install TensorFlowTTS python package.
## Stack Trace
```
Running TensorFlow Graph Passes: 100%|ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 6/6 [00:00<00:00, 11.82 passes/s]
Converting TF Frontend ==> MIL Ops: 76%|ββββββββββββββββββββββββββββββββββββββ | 1198/1585 [00:00<00:00, 6877.63 ops/s]
Traceback (most recent call last):
File "convert_tts.py", line 17, in
model = ct.convert(fastspeech2_convert,
File "Python/3.9/lib/python/site-packages/coremltools/converters/_converters_entry.py", line 444, in convert
mlmodel = mil_convert(
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/converter.py", line 187, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/converter.py", line 211, in _mil_convert
proto, mil_program = mil_convert_to_proto(
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/converter.py", line 281, in mil_convert_to_proto
prog = frontend_converter(model, **kwargs)
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/converter.py", line 99, in __call__
return tf2_loader.load()
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/frontend/tensorflow/load.py", line 82, in load
program = self._program_from_tf_ssa()
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/frontend/tensorflow2/load.py", line 210, in _program_from_tf_ssa
return converter.convert()
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 474, in convert
self.convert_main_graph(prog, graph)
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 397, in convert_main_graph
outputs = convert_graph(self.context, graph, self.output_names)
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/frontend/tensorflow/convert_utils.py", line 191, in convert_graph
add_op(context, node)
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/frontend/tensorflow/ops.py", line 1262, in RealDiv
y = mb.cast(x=context[node.inputs[1]], dtype="fp32")
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/mil/ops/registry.py", line 176, in add_op
return cls._add_op(op_cls_to_add, **kwargs)
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/mil/builder.py", line 182, in _add_op
new_op.type_value_inference()
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/mil/operation.py", line 256, in type_value_inference
output_vals = self._auto_val(output_types)
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/mil/operation.py", line 380, in _auto_val
vals = self.value_inference()
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/mil/operation.py", line 113, in wrapper
return func(self)
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/mil/ops/defs/iOS15/elementwise_unary.py", line 870, in value_inference
return self.get_cast_value(self.x, self.dtype.val)
File "Python/3.9/lib/python/site-packages/coremltools/converters/mil/mil/ops/defs/iOS15/elementwise_unary.py", line 896, in get_cast_value
return input_var.val.astype(dtype=type_map[dtype_val])
AttributeError: 'float' object has no attribute 'astype'
```
## To Reproduce
You will need to install TensorFlowTTS python package from https://github.com/TensorSpeech/TensorFlowTTS
- Please add a minimal code example that can reproduce the error when running it.
```
from tensorflow_tts.inference import AutoConfig
from tensorflow_tts.inference import TFAutoModel
import coremltools as ct
if __name__ == "__main__":
fastspeech2_config = AutoConfig.from_pretrained("tensorspeech/tts-fastspeech2-ljspeech-en")
fastspeech2_convert = TFAutoModel.from_pretrained(
config=fastspeech2_config,
name="fastspeech2",
enable_tflite_convertible=True,
)
print(fastspeech2_convert.summary())
model = ct.convert(fastspeech2_convert,
source='tensorflow',
convert_to='mlprogram',
minimum_deployment_target=ct.target.iOS16)
```
```
Model: "fastspeech2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
embeddings (TFFastSpeechEmb multiple 844032
eddings)
encoder (TFFastSpeechEncode multiple 11814400
r)
length_regulator (TFFastSpe multiple 0
echLengthRegulator)
decoder (TFFastSpeechDecode multiple 12601216
r)
mel_before (Dense) multiple 30800
postnet (TFTacotronPostnet) multiple 4352400
f0_predictor (TFFastSpeechV multiple 493313
ariantPredictor)
energy_predictor (TFFastSpe multiple 493313
echVariantPredictor)
duration_predictor (TFFastS multiple 493313
peechVariantPredictor)
f0_embeddings (Conv1D) multiple 3840
dropout_32 (Dropout) multiple 0
energy_embeddings (Conv1D) multiple 3840
dropout_33 (Dropout) multiple 0
=================================================================
Total params: 31,130,467
Trainable params: 29,552,579
Non-trainable params: 1,577,888
_________________________________________________________________
```
## System environment (please complete the following information):
- coremltools version: 6.2
- OS (e.g. MacOS version or Linux type): MacOS 13.2
- Any other relevant version information (e.g. PyTorch or TensorFlow version): Python 3.9, Tensorflow 2.10
Contributor guide
Research direction
Start with the minimal TensorFlowTTS conversion example and inspect the TensorFlow converter path in frontend/tensorflow/ops.py, especially RealDiv, along with the cast handling in mil/ops/defs/iOS15/elementwise_unary.py. Done means the supplied FastSpeech2 conversion no longer raises the reported AttributeError during Core ML conversion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100