apple / apple/coremltools

Strange issue: Converting RealESRGAN to CoreML model, ML Program format performs significantly slower than Neural Network format

Open
#2,545 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

Hi everyone,

I'm currently learning about machine learning in the Apple ecosystem. I tried converting the [RealESRGAN](https://github.com/xinntao/Real-ESRGAN) super-resolution model to a CoreML model and found that the ML Program format performs much slower than the Neural Network format, especially with load times being over 10 times longer.

Image

Image

In theory, the newer ML Program format should perform better, but that's not the case here. I'm not sure where the issue lies. Here's the [script](https://gist.github.com/itcook/e39f921eb50913988f4f762f33a80d8c) I used for the conversion—I'd really appreciate any help if someone can provide insights.

Additionally, after setting the input tensor's width and height to fixed dimensions, the model's tracing time became extremely long, whereas previously, using RangDim to set flexible shapes didn't exhibit this issue (the sample inputs were almost identical). This also puzzles me...

Furthermore, does a model with fixed-shape inputs generally perform better than one with flexible shapes? My understanding is that, in terms of performance: fixed-shape input > enumerated fixed-shapes input > flexible-shape input. Is this correct?

Contributor guide

Open the contributing guide

Research direction

Begin with the linked conversion script and its fixed- versus flexible-shape settings; compare the generated ML Program and Neural Network models using the reported load, tracing, and inference measurements. Done means identifying whether the slowdown is expected or a reproducible coremltools issue and documenting the conditions, since no repository test or source file is named.

Written by the indexing model from the issue text.

Assessment

Tech stack
machine-learning, python
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.