microsoft / microsoft/onnxruntime
[Performance] ONNX runtime takes too much CPU and RAM and slows the entire PC
@yihonglyu is already working on this.
Since Apr 9, 2024.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
I've converted a paddleOCR model into an onnx model , the model is around 9Mb and I run it on the CPU , but whenever I activate the onnx runtime to run the code it takes all the remaining CPU resources of the PC and a large chunk of the Ram.
### To reproduce
I'm pretty sure it has to do with the input shape logic , I've tried both static and dynamic input convergion . I've tried the onnx exporting tools Paddle2onnx and PaddleOCRModelConvert
this is the script i'm using to test the runtime.
```
ort_session = rt.InferenceSession('model.onnx')
so = rt.SessionOptions()
print(ort_session.get_inputs()[0].shape)
print(ort_session.get_outputs())
def preprocess_image(image_path):
image = Image.open(image_path).convert('RGB')
# image = image.resize((1000, 1000))
image_array = np.array(image)
image_array = image_array / 255.0
image_array = np.expand_dims(image_array, axis=0)
image_array = np.transpose(image_array, (1, 3, 0, 2))
return image_array
def get_memory_usage():
process = psutil.Process()
return process.memory_info().rss / (1024 ** 2) # Memory usage in MB
# List of image paths
image_paths = ['./ppocr_img/imgs_en/img_12.jpg']*50
memory_usage_list = [get_memory_usage()]
img_path = image_paths[0]
#for img_path in image_paths:
memory_before = get_memory_usage()
input_data = preprocess_image(img_path)
print("Input data shape:", input_data.shape)
ort_outputs = ort_session.run(None, {ort_session.get_inputs()[0].name: input_data.astype(np.float32)})[0]
memory_after = get_memory_usage()
memory_usage_list.append(memory_after)
```
### Urgency
Not urgent since it's a solo project but I spent a lot of time on this and just want to get over with it.
### Platform
Windows
### OS Version
11
### ONNX Runtime Installation
Released Package
### ONNX Runtime Version or Commit ID
1.9.0
### ONNX Runtime API
Python
### Architecture
X64
### Execution Provider
Default CPU
### Execution Provider Library Version
_No response_
### Model File
https://drive.google.com/file/d/1tCa3qzHzHEGq_KqzpAdCt5VLH-4Ym58k/view?usp=sharing
### Is this a quantized model?
Yes
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.