apple / apple/coremltools

Flexible input for MelSpectrogram: 'complex_shape' object has no attribute 'data'

Open
#2,211 0 comments 0 reactions 0 assignees View on GitHub
bug Flexible Shape PyTorch (traced) triaged
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describing the bug
CoreMLTools is able to convert `torchaudio.MelSpectrum` for a fixed input shape but it seems that it fails when the input shape is either an `RangeDim` or a `EnumeratedShapes`.

## Stack Trace
```
ERROR - 'common::lower_complex_dialect_ops' graph pass produces the following error:

Running MIL default pipeline: 0%| | 0/78 [00:00
core_model = coremltools.convert(
File "/Users/twoertwein/miniforge3/lib/python3.10/site-packages/coremltools/converters/_converters_entry.py", line 581, in convert
mlmodel = mil_convert(
File "/Users/twoertwein/miniforge3/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 188, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
File "/Users/twoertwein/miniforge3/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 212, in _mil_convert
proto, mil_program = mil_convert_to_proto(
File "/Users/twoertwein/miniforge3/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 291, in mil_convert_to_proto
PassPipelineManager.apply_pipeline(prog, main_pipeline)
File "/Users/twoertwein/miniforge3/lib/python3.10/site-packages/coremltools/converters/mil/mil/passes/pass_pipeline.py", line 478, in apply_pipeline
raise e # re-raise exception
File "/Users/twoertwein/miniforge3/lib/python3.10/site-packages/coremltools/converters/mil/mil/passes/pass_pipeline.py", line 473, in apply_pipeline
graph_pass(prog)
File "/Users/twoertwein/miniforge3/lib/python3.10/site-packages/coremltools/converters/mil/mil/passes/graph_pass.py", line 56, in __call__
self.apply(prog)
File "/Users/twoertwein/miniforge3/lib/python3.10/site-packages/coremltools/converters/mil/mil/passes/defs/lower_complex_dialect_ops.py", line 676, in apply
_lower_complex_dialect_ops_in_block(block)
File "/Users/twoertwein/miniforge3/lib/python3.10/site-packages/coremltools/converters/mil/mil/passes/helper.py", line 64, in wrapper
return _func(*args)
File "/Users/twoertwein/miniforge3/lib/python3.10/site-packages/coremltools/converters/mil/mil/passes/defs/lower_complex_dialect_ops.py", line 643, in _lower_complex_dialect_ops_in_block
_match_and_replace_dialect_op(block, op)
File "/Users/twoertwein/miniforge3/lib/python3.10/site-packages/coremltools/converters/mil/mil/passes/defs/lower_complex_dialect_ops.py", line 628, in _match_and_replace_dialect_op
lower_res = LowerComplex.get_lower_func(op.op_type)(op)
File "/Users/twoertwein/miniforge3/lib/python3.10/site-packages/coremltools/converters/mil/mil/passes/defs/lower_complex_dialect_ops.py", line 616, in _lower_complex_shape
return mb.shape(x=op.data.real, before_op=op)
AttributeError: 'complex_shape' object has no attribute 'data'
```

## To Reproduce
```py
import torch
import torchaudio
import coremltools

class Model(torch.nn.Module):
def __init__(self):
super().__init__()
self.mel_spectrum = torchaudio.transforms.MelSpectrogram(
power=1, sample_rate=16_000, n_mels=32
)

def forward(self, x):
return self.mel_spectrum(x)

x = torch.zeros(1_600)

model = Model()
model.eval()
model = torch.jit.trace(model, x)

# works
shape = x.shape
# fails
shape = [coremltools.RangeDim(lower_bound=1_600, upper_bound=16_000)]
# fails
shape = coremltools.EnumeratedShapes(shapes=[[1_600 * i] for i in range(1, 11)], default=[16_000])

coremltools.convert(
model,
convert_to="mlprogram",
inputs=[coremltools.TensorType(shape=shape)],
)
```

## System environment (please complete the following information):
- coremltools version: 72
- OS (e.g. MacOS version or Linux type): MacOs 14.4.1 (M2)
- Any other relevant version information: torch 2.2.0

Contributor guide

Open the contributing guide

Research direction

Start with mil/passes/defs/lower_complex_dialect_ops.py, especially _lower_complex_shape, and reproduce the failure using the provided MelSpectrogram conversion with RangeDim and EnumeratedShapes inputs. Done means conversion succeeds for both flexible input-shape cases without the complex_shape AttributeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.