Question about Converting Sander-wood (a Seq2Seq model) to Coreml
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## ❓Question
- If this is a question about the Core ML Frame work or Xcode, please ask your question in the Apple Developer Forum: https://developer.apple.com/forums/
I am currently trying to convert Sander-wood to coreml. BUT I met the problem when infer using mlpackage model. Here is the code I used to convert.
My question is, when I convert it using flexible inputs (actually should use it) and use this model to infer, always told me even if I generate the inputs exactly the same as the shape of model required.
**NSLocalizedDescription = "Unable to compute the prediction using a neural network model. It can be an invalid input data or broken/unsupported model (error code: -7).";**
After that I try to fix the shape of inputs when convert. It works. Does anyone know why this happened, or just because I input the wrong shape? though, I don't think so. I also paste the input shape and flexible shape that model accepted below.
```
class FullModelWrapper(torch.nn.Module):
def __init__(self, model):
super().__init__()
self.model = model
def forward(self, input_ids, decoder_input_ids):
outputs = self.model(
input_ids=input_ids,
decoder_input_ids=decoder_input_ids,
return_dict=False
)
return outputs[0]
def convert_whole_sanderwood_coreml(model_path):
tokenizer = AutoTokenizer.from_pretrained('/Users/maxfr/all-python-project/on-device-musc-generator/original')
model = AutoModelForSeq2SeqLM.from_pretrained('/Users/maxfr/all-python-project/on-device-musc-generator/original',
return_dict=False,
torchscript=True)
model.eval()
config = model.config
text = "This is a traditional Irish dance music."
inputs = tokenizer(text, return_tensors='pt', truncation=True, max_length=1024)
input_ids = inputs['input_ids'].to(torch.int32)
decoder_input_ids = torch.full((1, 10), model.config.decoder_start_token_id, dtype=torch.int32) # 形状 (1,10)
#decoder_input_ids = torch.tensor([[config.decoder_start_token_id]], dtype=torch.int32)
with torch.no_grad():
traced_model = torch.jit.trace(FullModelWrapper(model).eval(),
(input_ids,decoder_input_ids),
check_trace=True,
strict=False)
"""wrapped_model = FullModelWrapper(model)
dynamic_shapes = {
"input_ids": {1: torch.export.Dim("seq_len", min=1, max=1024)},
"decoder_input_ids": {1: torch.export.Dim("seq_len", min=1, max=1024)}
}
exported_model = torch.export.export(
wrapped_model,
(input_ids, decoder_input_ids),
dynamic_shapes=dynamic_shapes
)"""
input_shape = ct.Shape(shape=(1,
ct.RangeDim(lower_bound=1, upper_bound=1024, default=10)
))
coreml_model = ct.convert(
traced_model,
inputs=[
ct.TensorType(
name="input_ids",
shape=input_shape,
dtype=np.int32
),
ct.TensorType(
name="decoder_input_ids",
shape=input_shape,
dtype=np.int32
),
],
outputs=[
ct.TensorType(
name="logits",
)
],
minimum_deployment_target=ct.target.iOS16,
compute_units=ct.ComputeUnit.CPU_ONLY,
convert_to="mlprogram",
)
coreml_model.save(model_path)
print(f"Model successfully converted and saved to {model_path}")
```
the shape model accepted
default shape: [1, 10]
range: ['[1, 1]', '[1, 1024]']
default shape: [1, 10]
range: ['[1, 1]', '[1, 1024]']
the shape of my inputs
inputs input_ids: shape (1, 10), type int32
inputs decoder_input_ids: shape (1, 10), type int32
Contributor guide
Research direction
Start with the convert_whole_sanderwood_coreml function and reproduce the conversion using the shown traced model, flexible RangeDim inputs, and int32 shapes. Compare inference with the fixed-shape conversion; done means identifying whether the displayed shapes are sufficient to reproduce error -7 and documenting the missing condition or a minimal reproducible case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100