apple / apple/coremltools

PyTorch Tensor Slicing Assignment Bug

Open
#1,422 3 comments 4 reactions 0 assignees View on GitHub
bug PyTorch (not traced) triaged
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

Hi,

I have 2 bugs related to slicing.

1. In the `forward` method of a torch module, that uses rank 4 tensors, attempting to slice and assign to a number.
2. In the `forward` method of a torch module, that uses rank 4 tensors, attempting to slice and assign to a new tensor.

## Trace
1.
```
Traceback (most recent call last):
File "/home/guests//", line 22, in
mlmodel = coremltools.converters.convert(
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/_converters_entry.py", line 326, in convert
mlmodel = mil_convert(
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/converter.py", line 182, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/converter.py", line 209, in _mil_convert
proto, mil_program = mil_convert_to_proto(
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/converter.py", line 300, in mil_convert_to_proto
prog = frontend_converter(model, **kwargs)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/converter.py", line 104, in __call__
return load(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 50, in load
return _perform_torch_convert(converter, debug)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 87, in _perform_torch_convert
prog = converter.convert()
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 239, in convert
convert_nodes(self.context, self.graph)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 76, in convert_nodes
add_op(context, node)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 2689, in _internal_tensor_value_assign
updated_x = mb.torch_tensor_assign(
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/mil/ops/registry.py", line 63, in add_op
return cls._add_op(op_cls, **kwargs)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/mil/builder.py", line 191, in _add_op
new_op.type_value_inference()
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/mil/operation.py", line 240, in type_value_inference
output_types = self.type_inference()
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/frontend/torch/dialect_ops.py", line 220, in type_inference
raise ValueError("The updates tensor should have shape {}. Got {}".format(expected_updates_shape, self.updates.shape))
ValueError: The updates tensor should have shape (1, 12, 256, 256). Got (1, 12, 128, 256)
```
2.
```
Traceback (most recent call last):
File "/home/guests/", line 25, in
mlmodel = coremltools.converters.convert(
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/_converters_entry.py", line 326, in convert
mlmodel = mil_convert(
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/converter.py", line 182, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/converter.py", line 209, in _mil_convert
proto, mil_program = mil_convert_to_proto(
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/converter.py", line 300, in mil_convert_to_proto
prog = frontend_converter(model, **kwargs)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/converter.py", line 104, in __call__
return load(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 50, in load
return _perform_torch_convert(converter, debug)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 87, in _perform_torch_convert
prog = converter.convert()
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 239, in convert
convert_nodes(self.context, self.graph)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 76, in convert_nodes
add_op(context, node)
File "/usr/local/lib/python3.9/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 3441, in zeros
dtype = inputs[1].val
AttributeError: 'NoneType' object has no attribute 'val'
```

## To Reproduce
1.
```python
import torch
import coremltools

class Model(torch.nn.Module):

def __init__(self):
super().__init__()

def forward(self, input):
input[:,:,0::2,:] = 1
input[:,:,1::2,:] = 2

return input

if __name__ == "__main__":

model = Model()
input = torch.randn((1,12,256,256))

torchscript_model = torch.jit.script(model)

mlmodel = coremltools.converters.convert(
torchscript_model,
inputs=[coremltools.TensorType(name=f'input_0', shape=input.shape)],
minimum_deployment_target=coremltools.target.iOS14,
)
```

2.
```python
import torch
import coremltools

class Model(torch.nn.Module):

def __init__(self):
super().__init__()

def forward(self, input):

b, c, h, w = input.shape
xl = torch.zeros((b,c,h//2,w))

input[:,:,0::2,:] = xl

return input

if __name__ == "__main__":

model = Model()
input = torch.randn((1,12,256,256))

torchscript_model = torch.jit.script(model)

mlmodel = coremltools.converters.convert(
torchscript_model,
inputs=[coremltools.TensorType(name=f'input_0', shape=input.shape)],
minimum_deployment_target=coremltools.target.iOS14,
)
```

## System environment:
- coremltools==5.1.0
- torch==1.11
- OS: Linux
- How you install python: from source
- python version: 3.9.7

Contributor guide

Open the contributing guide

Research direction

Reproduce both TorchScript conversion cases from the issue, then inspect coremltools/converters/mil/frontend/torch/ops.py and dialect_ops.py at the reported assignment, zeros, and tensor type-inference paths. Done means both rank-4 slicing assignments convert without the reported shape or NoneType errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.