apple / apple/coremltools

MLState with flexible shape

Open
#2,537 1 comment 0 reactions 0 assignees View on GitHub
feature request
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🌱 Describe your Feature Request
Currently MLState does not support flexible input shape. Are we going to get support for MLStates that has dynamic shape?

## How can this feature be used?
This is an important feature for model that has large context window.

## Additional context
Example code to produce the "Flexible shape model states are not supported!" error:

```python
import numpy as np
import torch
import coremltools as ct

class Model(torch.nn.Module):
def __init__(self):
super().__init__()
self.register_buffer("accumulator", torch.zeros((1, 1), dtype=torch.float32))

def forward(self, x):
return self.accumulator * self.accumulator

state_shapes = ct.EnumeratedShapes(
shapes=[[1, 1], [2, 2], [3, 3]],
default=[1, 1]
)

traced_model = torch.jit.trace(Model().eval(), torch.ones(1, 1))

mlmodel = ct.convert(
traced_model,
inputs=[ct.TensorType(shape=(1, 1), name="x")],
outputs=[ct.TensorType(name="y")],
states=[
ct.StateType(
wrapped_type=ct.TensorType(
shape=state_shapes, # Using enumerated shapes for state
),
name="accumulator",
),
],
minimum_deployment_target=ct.target.iOS18,
)
mlmodel.save("model_state_enumerated.mlpackage")

```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the provided model through ct.convert with ct.StateType, ct.EnumeratedShapes, and the iOS 18 deployment target. Trace where conversion rejects flexible MLState shapes, then define and verify successful conversion for the shown example and its enumerated state shapes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.