converting 'cat' operation
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## πDescribing the bug
- Make sure to only create an issue here for bugs in the coremltools Python package. If this is a bug with the Core ML Framework or Xcode, please submit your bug here: https://developer.apple.com/bug-reporting/
- Provide a clear and consise description of the bug.
While converting from detectron2 model to coreml, got the following error:
ValueError: dtypes needs to be a list/tuple of at least 1 element
## Stack Trace
ERROR - converting 'cat' op (located at: 'roi_heads/box_pooler'):
Converting PyTorch Frontend ==> MIL Ops: 75%|βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | 1426/1891 [00:00<00:00, 1685.47 ops/s]
Traceback (most recent call last):
File "/home/user/Desktop/drive/segmentation/model/torch2coreml.py", line 12, in
mlmodel = ct.converters.convert(
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/_converters_entry.py", line 581, in convert
mlmodel = mil_convert(
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 188, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 212, in _mil_convert
proto, mil_program = mil_convert_to_proto(
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 288, in mil_convert_to_proto
prog = frontend_converter(model, **kwargs)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 108, in __call__
return load(*args, **kwargs)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 82, in load
return _perform_torch_convert(converter, debug)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 116, in _perform_torch_convert
prog = converter.convert()
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 581, in convert
convert_nodes(self.context, self.graph)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 86, in convert_nodes
raise e # re-raise exception
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 81, in convert_nodes
convert_single_node(context, node)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 134, in convert_single_node
add_op(context, node)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 2115, in cat
values=promote_input_dtypes(xs), axis=axis, name=node.name
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/mil/ops/defs/_utils.py", line 452, in promote_input_dtypes
promoted_dtype = promote_dtypes([var.dtype for var in input_vars])
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/mil/types/type_mapping.py", line 264, in promote_dtypes
raise ValueError("dtypes needs to be a list/tuple of at least 1 element")
ValueError: dtypes needs to be a list/tuple of at least 1 element
## System environment (please complete the following information):
- coremltools version: 7.2
- OS (e.g. MacOS version or Linux type): Ubuntu 20.04.6 LTS
- Any other relevant version information (e.g. PyTorch or TensorFlow version): torch==2.2.0, detectron2==0.6
## Additional context
- also initially i was getting a `KeyError: 4` for the full operation, to counter that, i copied code from the latest beta release for the full operation. the copied code:
```
@register_torch_op
def full(context, node):
inputs = _get_inputs(context, node, min_expected=2)
size = inputs[0]
# dtype could be torch.dtype or an integer that maps to a numpy.dtype
dtype = None
if len(inputs) < 3 or inputs[2] is None:
dtype = np.float32
elif isinstance(inputs[2].val, torch.dtype):
dtype = NUM_TO_NUMPY_DTYPE[TORCH_DTYPE_TO_NUM[inputs[2].val]]
elif isinstance(inputs[2].val, (int, np.generic)):
dtype = NUM_TO_NUMPY_DTYPE[inputs[2].val]
else:
raise ValueError(f"unsupported type {type(inputs[2].val)}.")
val = dtype(inputs[1].val)
result = _make_fill_op(size, val, node.name)
context.add(result)
```
Contributor guide
Research direction
Start by reproducing the Detectron2 conversion with coremltools 7.2, then inspect coremltools/converters/mil/frontend/torch/ops.py at the cat handler and the dtype helpers in coremltools/converters/mil/mil/ops/defs/_utils.py and type_mapping.py. Trace why the cat inputs produce an empty dtype list and check the related full-operation handling. Done means the reported conversion no longer raises this error and regression coverage verifies the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100