Stateful MIL to CoreML breaks when fixed and flexible inputs are present
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## 🐞Describing the bug
Using a mix of static and range input shapes with a stateful model fails.
While trying to make a minimal repro, I encountered some other strange errors with static shapes, range shapes, and state interacting to fail with odd combinations.
## Stack Trace
```
libc++abi: terminating due to uncaught exception of type CoreML::MLNeuralNetworkUtilities::EnumeratedWithRangeInputsException: A model doesn't allow a mixture of enumerated and range shape flexibility, but feature (fixed_tensor) uses the enumerated shape and feature (flex_tensor) uses the range shape.
```
## To Reproduce
This will reproduce the error. As written, the good version run and the final version fails.
In comments, I also have some variations that seem like they should be an even more minimal repro, but fail for unknown (different) reasons.
```python
import numpy as np
import coremltools as ct
from coremltools.converters.mil.mil import Builder as mb
import coremltools.converters.mil.mil.types as mil_types
# Input shapes
# fixed tensor
fixed_tensor_spec = mb.TensorSpec(shape=(5,), dtype=mil_types.fp32)
fixed_tensor_type = ct.TensorType(shape=(5,), dtype=mil_types.fp32, name='fixed_tensor')
# flex tensor
flex_dim = ct.RangeDim(lower_bound=1, upper_bound=1024)
flex_tensor_spec = mb.TensorSpec(shape=(flex_dim.symbol,), dtype=mil_types.fp32)
flex_tensor_type = ct.TensorType(shape=(flex_dim,), dtype=mil_types.fp32, name='flex_tensor')
# state
fixed_state_spec = mb.StateTensorSpec(
shape=(32,),
dtype=mil_types.fp16
)
# This program works fine (mix of static and range shapes, no state)
@mb.program(input_specs=[fixed_tensor_spec, flex_tensor_spec], opset_version=ct.target.iOS18)
def good_prog_stateless(fixed_tensor,
flex_tensor,
):
return (
mb.identity(x=fixed_tensor),
mb.identity(x=flex_tensor),
)
ct_good_prog_stateless = ct.convert(
good_prog_stateless,
inputs=[fixed_tensor_type, flex_tensor_type],
minimum_deployment_target=ct.target.iOS18,
)
ct_good_prog_stateless.predict({
"fixed_tensor": np.random.randn(5).astype(np.float32),
"flex_tensor": np.random.randn(7).astype(np.float32),
}) # successful
# This program works fine (range shape + fixed state)
# BUT it fails if we don't operate on the input and the state
@mb.program(input_specs=[flex_tensor_spec, fixed_state_spec], opset_version=ct.target.iOS18)
def good_prog_flex_stateful(flex_tensor,
fixed_state,
):
fixed_state_val = mb.read_state(input=fixed_state)
# This output fails:
# return (
# mb.identity(x=flex_tensor),
# mb.identity(x=fixed_state_val),
# )
# failure details:
# runtime warning at convert:
# RuntimeWarning: You will not be able to run predict() on this Core ML model. Underlying exception message was: {
# NSLocalizedDescription = "Failed to build the model execution plan using a model architecture file '/private/var/folders/sq/0rb5nqc14gdb4n4bvt37_tc00000gn/T/tmp8l6i1bqd.mlmodelc/model.mil' with error code: 14.";
# }
# then exception at .make_state:
# Traceback (most recent call last):
# File "/Users/ianhorn/Library/Application Support/JetBrains/PyCharm2025.1/scratches/coreml_shapes.py", line 64, in
# flex_state = ct_good_prog_flex_stateful.make_state()
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# File "/Users/ianhorn/code/hmer/modeling/.venv/lib/python3.12/site-packages/coremltools/models/model.py", line 884, in make_state
# raise Exception("This model was not loaded with the Core ML Framework. Cannot get state.")
# Exception: This model was not loaded with the Core ML Framework. Cannot get state.
# This output succeeds (both returns need to be changed from passthrough)
# Adding zero to the state works, but not to the input
state_adder = mb.const(val=np.array(0, dtype=np.float16))
return (
mb.add(x=flex_tensor, y=1.0), # but we have to add non-zero to the input for it to run
mb.add(x=fixed_state_val, y=state_adder),
)
ct_good_prog_flex_stateful = ct.convert(
good_prog_flex_stateful,
inputs=[flex_tensor_type],
minimum_deployment_target=ct.target.iOS18,
)
flex_state = ct_good_prog_flex_stateful.make_state()
ct_good_prog_flex_stateful.predict({
"flex_tensor": np.random.randn(7).astype(np.float32),
}, state=flex_state) # successful
# This program works fine (fixed shape + fixed state)
# BUT it fails if we don't operate on the input
@mb.program(input_specs=[fixed_tensor_spec, fixed_state_spec], opset_version=ct.target.iOS18)
def good_prog_fixed_stateful(fixed_tensor,
fixed_state,
):
fixed_state_val = mb.read_state(input=fixed_state)
# This output fails:
# return (
# mb.identity(x=fixed_tensor),
# mb.identity(x=fixed_state_val),
# )
# failure details:
# same as good_prog_flex_stateful above
# This output succeeds (only the input needs to be changed from passthrough)
# notice the difference from `good_prog_flex_stateful`: When the *input* tensor has fixed
# shape, the *state* tensor doesn't need to be touched for it to compile + run.
return (
mb.add(x=fixed_tensor, y=1.0),
mb.identity(x=fixed_state_val),
)
ct_good_prog_fixed_stateful = ct.convert(
good_prog_fixed_stateful,
inputs=[flex_tensor_type],
minimum_deployment_target=ct.target.iOS18,
)
fixed_state = ct_good_prog_fixed_stateful.make_state()
ct_good_prog_fixed_stateful.predict({
"fixed_tensor": np.random.randn(5).astype(np.float32),
}, state=fixed_state) # successful
# This program fails (fixed shape + range shape + fixed state)
@mb.program(input_specs=[fixed_tensor_spec, flex_tensor_spec, fixed_state_spec], opset_version=ct.target.iOS18)
def bad_prog_fixed_and_flex_stateful(fixed_tensor,
flex_tensor,
fixed_state,
):
fixed_state_val = mb.read_state(input=fixed_state)
# As above, it fails if we try to return the inputs untouched:
# return (
# mb.identity(x=fixed_tensor),
# mb.identity(x=flex_tensor),
# mb.identity(x=fixed_state_val),
# )
# failure details:
# same as `good_prog_flex_stateful` and `good_prog_flex_stateful` above
# It appears to compile correctly when we touch each of the inputs
# but it fails on .make_state() detailed below
state_adder = mb.const(val=np.array(0, dtype=np.float16))
return (
mb.add(x=fixed_tensor, y=1.0),
mb.add(x=flex_tensor, y=1.0),
mb.add(x=fixed_state_val, y=state_adder),
)
ct_bad_prog = ct.convert(
bad_prog_fixed_and_flex_stateful,
inputs=[fixed_tensor_type, flex_tensor_type],
minimum_deployment_target=ct.target.iOS18,
)
state = ct_bad_prog.make_state() # fails:
# libc++abi: terminating due to uncaught exception of type CoreML::MLNeuralNetworkUtilities::EnumeratedWithRangeInputsException: A model doesn't allow a mixture of enumerated and range shape flexibility, but feature (fixed_tensor) uses the enumerated shape and feature (flex_tensor) uses the range shape.
#
# Process finished with exit code 134 (interrupted by signal 6:SIGABRT)
```
## System environment (please complete the following information):
- coremltools version: 8.3.0
- OS (e.g. MacOS version or Linux type): 15.1 (24B83)
## Additional context
- I'd love to know if I'm just not doing things idiomatically. I moved from `torch --> coreml` conversion to `hand written MIL --> coreml` because the torch conversions were too buggy, but MIL isn't very heavily documented, so IDK if i'm just using it wrong.
Contributor guide
Research direction
Start with the four MIL entry points named in the reproducer: good_prog_stateless, good_prog_flex_stateful, good_prog_fixed_stateful, and bad_prog_fixed_and_flex_stateful. Trace their ct.convert(), make_state(), and predict() calls on coremltools 8.3.0, comparing the successful and failing shape/state combinations. Done means the mixed fixed-and-range stateful case can create state and run without the EnumeratedWithRangeInputsException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- machine-learning, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100