Converting RTDETR Object Detection Model to CoreML
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
I am trying to convert the RTDETR Object Detection model to CoreML for use in an iOS app. However, I am encountering a data type mismatch error during the conversion process.
The code i am using:
```
import torch
import coremltools as ct
from transformers import RTDetrForObjectDetection
model = RTDetrForObjectDetection.from_pretrained("TMP-finetune/runs/Jan22_16-22-29")
example_input = (torch.rand(1, 3, 512, 512),)
# Generate ExportedProgram by exporting
exported_model = torch.export.export(model, example_input)
input_images = ct.ImageType(name="images", shape=example_input[0].shape)
# Convert the model
mlmodel = ct.convert(
exported_model,
inputs=[input_images]
)
# Save the Core ML model
mlmodel.save("best.mlmodel")
```
The error i am getting:

I have tried searching for solutions but haven't found a way to resolve this data type mismatch issue. Any guidance or suggestions would be greatly appreciated.
I have also opened a related issue on another repository, which can be found here: https://github.com/huggingface/transformers/issues/35905#issuecomment-2636625966
Thanks in advance
Contributor guide
Research direction
Start by reproducing the failure in the shown torch.export.export and ct.convert calls with RTDetrForObjectDetection, then inspect the reported data type mismatch. Compare the conversion behavior with the linked Transformers issue. Done means identifying a compatible conversion path and successfully saving best.mlmodel for iOS use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, mobile-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100