microsoft / microsoft/onnxruntime

[Performance] ONNX runtime takes too much CPU and RAM and slows the entire PC

Open
#20,243 4 comments 0 reactions 1 assignee View on GitHub

@yihonglyu is already working on this.

Since Apr 9, 2024.

performance platform:windows quantization
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

I've converted a paddleOCR model into an onnx model , the model is around 9Mb and I run it on the CPU , but whenever I activate the onnx runtime to run the code it takes all the remaining CPU resources of the PC and a large chunk of the Ram.

### To reproduce

I'm pretty sure it has to do with the input shape logic , I've tried both static and dynamic input convergion . I've tried the onnx exporting tools Paddle2onnx and PaddleOCRModelConvert

this is the script i'm using to test the runtime.
```
ort_session = rt.InferenceSession('model.onnx')
so = rt.SessionOptions()

print(ort_session.get_inputs()[0].shape)
print(ort_session.get_outputs())

def preprocess_image(image_path):
image = Image.open(image_path).convert('RGB')
# image = image.resize((1000, 1000))
image_array = np.array(image)
image_array = image_array / 255.0
image_array = np.expand_dims(image_array, axis=0)
image_array = np.transpose(image_array, (1, 3, 0, 2))
return image_array

def get_memory_usage():
process = psutil.Process()
return process.memory_info().rss / (1024 ** 2) # Memory usage in MB

# List of image paths
image_paths = ['./ppocr_img/imgs_en/img_12.jpg']*50
memory_usage_list = [get_memory_usage()]
img_path = image_paths[0]
#for img_path in image_paths:

memory_before = get_memory_usage()

input_data = preprocess_image(img_path)
print("Input data shape:", input_data.shape)
ort_outputs = ort_session.run(None, {ort_session.get_inputs()[0].name: input_data.astype(np.float32)})[0]
memory_after = get_memory_usage()
memory_usage_list.append(memory_after)
```

### Urgency

Not urgent since it's a solo project but I spent a lot of time on this and just want to get over with it.

### Platform

Windows

### OS Version

11

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.9.0

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

Default CPU

### Execution Provider Library Version

_No response_

### Model File
https://drive.google.com/file/d/1tCa3qzHzHEGq_KqzpAdCt5VLH-4Ym58k/view?usp=sharing

### Is this a quantized model?

Yes

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.