microsoft / microsoft/onnxruntime-inference-examples

Throwing OutOfMemoryError FATAL EXCEPTION for bigger ONNX/ORT models on Android

Open
#183 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.7k
Forks
414
Avg merge
1d 6h
Merged PRs (30d)
14

Description

I'm following ONNX Runtime Mobile Examples on android & all are working well.

I'm using the same implementation to run my custom ONNX/ORT model on android but getting the following exception :

E/AndroidRuntime: FATAL EXCEPTION: main
Process: ai.onnxruntime.example.imageclassifier, PID: 21599
java.lang.OutOfMemoryError: Failed to allocate a 361332168 byte allocation with 8388608 free bytes and 164MB until OOM, target footprint 372920392, growth limit 536870912
at java.util.Arrays.copyOf(Arrays.java:3670)
at java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:211)
at kotlin.io.ByteStreamsKt.readBytes(IOStreams.kt:137)
at ai.onnxruntime.example.imageclassifier.MainActivity$readModel$2.invokeSuspend(MainActivity.kt:154)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Following are some development environment and model specifications I'm using :

  1. IDE - Android Studio Dolphin | 2021.3.1 Patch 1
  2. ONNX Library: com.microsoft.onnxruntime:onnxruntime-mobile:1.8.2
  3. Model: Human Pose Estimation - Vison Transformer (Model Size 340 MB)

Is there a bigger model size creating the problem?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Android example's MainActivity.kt at line 154 and the readModel coroutine, where the stack trace shows the model is read into a byte array. Reproduce the failure with the reported 340 MB Human Pose Estimation Vision Transformer model and compare it with the working examples. Done means documenting whether the model size or loading approach causes the allocation failure and identifying a supported path.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, kotlin
Domain
machine-learning, mobile-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.