microsoft / microsoft/onnxruntime-inference-examples
Throwing OutOfMemoryError FATAL EXCEPTION for bigger ONNX/ORT models on Android
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.7k
- Forks
- 414
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 14
Description
I'm following ONNX Runtime Mobile Examples on android & all are working well.
I'm using the same implementation to run my custom ONNX/ORT model on android but getting the following exception :
E/AndroidRuntime: FATAL EXCEPTION: main
Process: ai.onnxruntime.example.imageclassifier, PID: 21599
java.lang.OutOfMemoryError: Failed to allocate a 361332168 byte allocation with 8388608 free bytes and 164MB until OOM, target footprint 372920392, growth limit 536870912
at java.util.Arrays.copyOf(Arrays.java:3670)
at java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:211)
at kotlin.io.ByteStreamsKt.readBytes(IOStreams.kt:137)
at ai.onnxruntime.example.imageclassifier.MainActivity$readModel$2.invokeSuspend(MainActivity.kt:154)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Following are some development environment and model specifications I'm using :
- IDE - Android Studio Dolphin | 2021.3.1 Patch 1
- ONNX Library: com.microsoft.onnxruntime:onnxruntime-mobile:1.8.2
- Model: Human Pose Estimation - Vison Transformer (Model Size 340 MB)
Is there a bigger model size creating the problem?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Android example's MainActivity.kt at line 154 and the readModel coroutine, where the stack trace shows the model is read into a byte array. Reproduce the failure with the reported 340 MB Human Pose Estimation Vision Transformer model and compare it with the working examples. Done means documenting whether the model size or loading approach causes the allocation failure and identifying a supported path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, kotlin
- Domain
- machine-learning, mobile-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100