MLState with flexible shape
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## 🌱 Describe your Feature Request
Currently MLState does not support flexible input shape. Are we going to get support for MLStates that has dynamic shape?
## How can this feature be used?
This is an important feature for model that has large context window.
## Additional context
Example code to produce the "Flexible shape model states are not supported!" error:
```python
import numpy as np
import torch
import coremltools as ct
class Model(torch.nn.Module):
def __init__(self):
super().__init__()
self.register_buffer("accumulator", torch.zeros((1, 1), dtype=torch.float32))
def forward(self, x):
return self.accumulator * self.accumulator
state_shapes = ct.EnumeratedShapes(
shapes=[[1, 1], [2, 2], [3, 3]],
default=[1, 1]
)
traced_model = torch.jit.trace(Model().eval(), torch.ones(1, 1))
mlmodel = ct.convert(
traced_model,
inputs=[ct.TensorType(shape=(1, 1), name="x")],
outputs=[ct.TensorType(name="y")],
states=[
ct.StateType(
wrapped_type=ct.TensorType(
shape=state_shapes, # Using enumerated shapes for state
),
name="accumulator",
),
],
minimum_deployment_target=ct.target.iOS18,
)
mlmodel.save("model_state_enumerated.mlpackage")
```
Contributor guide
Research direction
Start by reproducing the provided model through ct.convert with ct.StateType, ct.EnumeratedShapes, and the iOS 18 deployment target. Trace where conversion rejects flexible MLState shapes, then define and verify successful conversion for the shown example and its enumerated state shapes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100