apple / apple/coremltools

Question about Converting Sander-wood (a Seq2Seq model) to Coreml

Open
#2,463 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## ❓Question

- If this is a question about the Core ML Frame work or Xcode, please ask your question in the Apple Developer Forum: https://developer.apple.com/forums/

I am currently trying to convert Sander-wood to coreml. BUT I met the problem when infer using mlpackage model. Here is the code I used to convert.
My question is, when I convert it using flexible inputs (actually should use it) and use this model to infer, always told me even if I generate the inputs exactly the same as the shape of model required.
**NSLocalizedDescription = "Unable to compute the prediction using a neural network model. It can be an invalid input data or broken/unsupported model (error code: -7).";**

After that I try to fix the shape of inputs when convert. It works. Does anyone know why this happened, or just because I input the wrong shape? though, I don't think so. I also paste the input shape and flexible shape that model accepted below.

```
class FullModelWrapper(torch.nn.Module):
def __init__(self, model):
super().__init__()
self.model = model

def forward(self, input_ids, decoder_input_ids):

outputs = self.model(
input_ids=input_ids,
decoder_input_ids=decoder_input_ids,
return_dict=False
)
return outputs[0]

def convert_whole_sanderwood_coreml(model_path):
tokenizer = AutoTokenizer.from_pretrained('/Users/maxfr/all-python-project/on-device-musc-generator/original')
model = AutoModelForSeq2SeqLM.from_pretrained('/Users/maxfr/all-python-project/on-device-musc-generator/original',
return_dict=False,
torchscript=True)

model.eval()
config = model.config

text = "This is a traditional Irish dance music."
inputs = tokenizer(text, return_tensors='pt', truncation=True, max_length=1024)
input_ids = inputs['input_ids'].to(torch.int32)

decoder_input_ids = torch.full((1, 10), model.config.decoder_start_token_id, dtype=torch.int32) # 形状 (1,10)
#decoder_input_ids = torch.tensor([[config.decoder_start_token_id]], dtype=torch.int32)


with torch.no_grad():
traced_model = torch.jit.trace(FullModelWrapper(model).eval(),
(input_ids,decoder_input_ids),
check_trace=True,
strict=False)



"""wrapped_model = FullModelWrapper(model)

dynamic_shapes = {
"input_ids": {1: torch.export.Dim("seq_len", min=1, max=1024)},
"decoder_input_ids": {1: torch.export.Dim("seq_len", min=1, max=1024)}
}

exported_model = torch.export.export(
wrapped_model,
(input_ids, decoder_input_ids),
dynamic_shapes=dynamic_shapes
)"""
input_shape = ct.Shape(shape=(1,
ct.RangeDim(lower_bound=1, upper_bound=1024, default=10)
))

coreml_model = ct.convert(
traced_model,
inputs=[
ct.TensorType(
name="input_ids",
shape=input_shape,
dtype=np.int32
),
ct.TensorType(
name="decoder_input_ids",
shape=input_shape,
dtype=np.int32
),
],
outputs=[
ct.TensorType(
name="logits",
)
],
minimum_deployment_target=ct.target.iOS16,
compute_units=ct.ComputeUnit.CPU_ONLY,
convert_to="mlprogram",
)

coreml_model.save(model_path)
print(f"Model successfully converted and saved to {model_path}")
```
the shape model accepted
default shape: [1, 10]
range: ['[1, 1]', '[1, 1024]']
default shape: [1, 10]
range: ['[1, 1]', '[1, 1024]']

the shape of my inputs
inputs input_ids: shape (1, 10), type int32
inputs decoder_input_ids: shape (1, 10), type int32

Contributor guide

Open the contributing guide

Research direction

Start with the convert_whole_sanderwood_coreml function and reproduce the conversion using the shown traced model, flexible RangeDim inputs, and int32 shapes. Compare inference with the fixed-shape conversion; done means identifying whether the displayed shapes are sufficient to reproduce error -7 and documenting the missing condition or a minimal reproducible case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.