Flexible size causes error for dynamic resizing
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## ❓Question
I tried this code,
```python
import pytorch_lightning as pl
import torch
import coremltools
from coremltools.models.neural_network import flexible_shape_utils
from torch import FloatTensor
import torch.nn as nn
class Decoder(pl.LightningModule):
def __init__(self):
super().__init__()
self.word_embed = nn.Sequential(
nn.Embedding(118, 256), nn.LayerNorm(256)
)
def forward(self, src: FloatTensor) -> FloatTensor:
# return torch.flatten(src, 0, 1)
return self.word_embed(src)
model = Decoder()
with torch.no_grad():
input_tensor = torch.ones([1, 10], dtype=torch.int32)
traced_model = torch.jit.trace(model.eval(), input_tensor, check_trace=True, check_tolerance=True)
coreml_model = coremltools.convert(
traced_model,
inputs=[coremltools.TensorType(shape=input_tensor.shape, dtype=int)]
)
coreml_model.save("decoder.mlmodel")
spec = coremltools.utils.load_spec('decoder.mlmodel')
input_name = spec.description.input[0].name
flexible_shape_utils.set_multiarray_ndshape_range(spec,
feature_name=input_name,
lower_bounds=[1, 5],
upper_bounds=[1, 10])
coremltools.utils.save_spec(spec, 'decoder.mlmodel')
loaded = coremltools.models.MLModel('decoder.mlmodel')
input_tensor = input_tensor.detach().numpy().copy()
out = loaded.predict({'src': input_tensor}) # this is OK
print(out)
input_tensor = torch.ones([1, 5], dtype=torch.int32).detach().numpy().copy()
out = loaded.predict({'src': input_tensor}) # error occured
print(out)
```
I got error.
```
Traceback (most recent call last):
File "/Users/ryosukefukatani/work/HMERModel/atnBTTR/d2.py", line 42, in
out = loaded.predict({'src': input_tensor})
File "/Users/ryosukefukatani/work/HMERModel/venv/lib/python3.9/site-packages/coremltools/models/model.py", line 512, in predict
return self.__proxy__.predict(data, useCPUOnly)
RuntimeError: {
NSLocalizedDescription = "Failure dynamically resizing for sequence length.";
}
```
When I removed `nn.LayerNorm(256)`, this code was passed.
How to fix it? or `LayerNorm` does not support flexible size?
### Environment
- macOS 12.1
- coremltools 5.2
- torch 1.10.2
- Python 3.9.6
Contributor guide
Research direction
Start by reproducing the supplied Python example with nn.LayerNorm, coremltools.convert, and flexible_shape_utils.set_multiarray_ndshape_range using input shapes [1, 10] and [1, 5]. Investigate the dynamic-resizing path involved in the LayerNorm-converted model; done means the shorter prediction succeeds or the limitation is clearly identified and documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100