apple / apple/coremltools

Error when converting a PyTorch traced model to “neuralnetwork” or “mlprogram" format

Open
#2,112 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describing the bug
I’m trying to create a coreml package from a traced model, but I get the following error when calling coremltools.convert:

ValueError: Op "135" (op_type: slice_by_index) Input x="130" expects tensor or scalar of dtype from type domain ['fp16', 'fp32', 'int32', 'bool'] but got tensor[1,2,2049,440,complex64]

## Stack Trace
Converting PyTorch Frontend ==> MIL Ops: 5%| | 91/1722 [00:00<00:00, 3237.72 o
Traceback (most recent call last):
File "/Users/andreascuderi/nTrack/trunk/n-Track_EX/Scripts/./hybrid_demucs_tracer.py", line 72, in
mlpackage_obj = ct.convert(
^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/_converters_entry.py", line 574, in convert
mlmodel = mil_convert(
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/converter.py", line 188, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/converter.py", line 212, in _mil_convert
proto, mil_program = mil_convert_to_proto(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/converter.py", line 286, in mil_convert_to_proto
prog = frontend_converter(model, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/converter.py", line 108, in __call__
return load(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 80, in load
return _perform_torch_convert(converter, debug)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 99, in _perform_torch_convert
prog = converter.convert()
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 519, in convert
convert_nodes(self.context, self.graph)
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 88, in convert_nodes
add_op(context, node)
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 4209, in _slice
res = mb.slice_by_index(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/mil/ops/registry.py", line 182, in add_op
return cls._add_op(op_cls_to_add, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/mil/builder.py", line 168, in _add_op
new_op = op_cls(**kwargs)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/mil/operation.py", line 190, in __init__
self._validate_and_set_inputs(input_kv)
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/mil/operation.py", line 503, in _validate_and_set_inputs
self.input_spec.validate_inputs(self.name, self.op_type, input_kvs)
File "/usr/local/lib/python3.11/site-packages/coremltools/converters/mil/mil/input_type.py", line 163, in validate_inputs
raise ValueError(msg.format(name, var.name, input_type.type_str,
ValueError: Op "138" (op_type: slice_by_index) Input x="133" expects tensor or scalar of dtype from type domain ['fp16', 'fp32', 'int32', 'bool'] but got tensor[1,2,2049,44,complex64]

## To Reproduce
input = torch.rand(1, 2, 40000)
traced_module = torch.jit.trace(model, input)
ct.convert(traced_module, convert_to="neuralnetwork”, inputs=[ct.TensorType(shape=input.shape)])

## System environment (please complete the following information):
- coremltools version: 7.1
- macOS 13.2.1
- PyTorch version: 2.1.0

## Additional context
- Add anything else about the problem here that you want to share.

Contributor guide

Open the contributing guide

Research direction

Start by running the traced-model reproduction with coremltools 7.1, then inspect coremltools/converters/mil/frontend/torch/ops.py at _slice and the MIL slice_by_index validation shown in the traceback. Determine how the complex64 tensor reaches this operation; done requires a confirmed supported conversion path or a regression test and clear handling of the unsupported case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.