apple / apple/coremltools

converting 'cat' operation

Open
#2,296 4 comments 0 reactions 0 assignees View on GitHub
bug triaged
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describing the bug
- Make sure to only create an issue here for bugs in the coremltools Python package. If this is a bug with the Core ML Framework or Xcode, please submit your bug here: https://developer.apple.com/bug-reporting/
- Provide a clear and consise description of the bug.
While converting from detectron2 model to coreml, got the following error:

ValueError: dtypes needs to be a list/tuple of at least 1 element

## Stack Trace
ERROR - converting 'cat' op (located at: 'roi_heads/box_pooler'):

Converting PyTorch Frontend ==> MIL Ops: 75%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ | 1426/1891 [00:00<00:00, 1685.47 ops/s]
Traceback (most recent call last):
File "/home/user/Desktop/drive/segmentation/model/torch2coreml.py", line 12, in
mlmodel = ct.converters.convert(
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/_converters_entry.py", line 581, in convert
mlmodel = mil_convert(
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 188, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 212, in _mil_convert
proto, mil_program = mil_convert_to_proto(
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 288, in mil_convert_to_proto
prog = frontend_converter(model, **kwargs)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 108, in __call__
return load(*args, **kwargs)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 82, in load
return _perform_torch_convert(converter, debug)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 116, in _perform_torch_convert
prog = converter.convert()
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 581, in convert
convert_nodes(self.context, self.graph)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 86, in convert_nodes
raise e # re-raise exception
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 81, in convert_nodes
convert_single_node(context, node)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 134, in convert_single_node
add_op(context, node)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 2115, in cat
values=promote_input_dtypes(xs), axis=axis, name=node.name
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/mil/ops/defs/_utils.py", line 452, in promote_input_dtypes
promoted_dtype = promote_dtypes([var.dtype for var in input_vars])
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/mil/types/type_mapping.py", line 264, in promote_dtypes
raise ValueError("dtypes needs to be a list/tuple of at least 1 element")
ValueError: dtypes needs to be a list/tuple of at least 1 element

## System environment (please complete the following information):
- coremltools version: 7.2
- OS (e.g. MacOS version or Linux type): Ubuntu 20.04.6 LTS
- Any other relevant version information (e.g. PyTorch or TensorFlow version): torch==2.2.0, detectron2==0.6

## Additional context
- also initially i was getting a `KeyError: 4` for the full operation, to counter that, i copied code from the latest beta release for the full operation. the copied code:

```
@register_torch_op
def full(context, node):
inputs = _get_inputs(context, node, min_expected=2)

size = inputs[0]

# dtype could be torch.dtype or an integer that maps to a numpy.dtype
dtype = None
if len(inputs) < 3 or inputs[2] is None:
dtype = np.float32
elif isinstance(inputs[2].val, torch.dtype):
dtype = NUM_TO_NUMPY_DTYPE[TORCH_DTYPE_TO_NUM[inputs[2].val]]
elif isinstance(inputs[2].val, (int, np.generic)):
dtype = NUM_TO_NUMPY_DTYPE[inputs[2].val]
else:
raise ValueError(f"unsupported type {type(inputs[2].val)}.")

val = dtype(inputs[1].val)

result = _make_fill_op(size, val, node.name)
context.add(result)
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Detectron2 conversion with coremltools 7.2, then inspect coremltools/converters/mil/frontend/torch/ops.py at the cat handler and the dtype helpers in coremltools/converters/mil/mil/ops/defs/_utils.py and type_mapping.py. Trace why the cat inputs produce an empty dtype list and check the related full-operation handling. Done means the reported conversion no longer raises this error and regression coverage verifies the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.