openvinotoolkit / openvinotoolkit/model_server

Memory leaks on infer requests

Open
#2,102 2 comments 0 reactions 1 assignee View on GitHub

@dkalinowski is already working on this.

Since Oct 25, 2023.

bug
Dominant language
C++
Stars
931
Forks
277
Avg merge
2d 13h
Merged PRs (30d)
68

Description

Describe the bug
The server leaks memory on infer requests.

To Reproduce
To reproduce

  1. Get the archive ov-leak-debug.tar.gz, it contains a load simulation script, docker-compose file to run the server, the models and the model_config.json, and a script that creates the models (for reference).
  2. Run the server with docker compose up -d.
  3. Install the dependencies pip install -r requirements.txt.
  4. Run the script python generate_ovms_load.py {model_name} --n-workers 10 --n-threads 10 with {model_name} being static, dynamic or dynamic-nms.
  5. Check OVMS memory usage to see it creep up. It fluctuates for static, creeps up slowly for dynamic, and grows rapidly for dynamic-nms. The memory usage doesn't go down even when the load is no longer applied.

Expected behavior
The memory used by OVMS is constant (or stabilizes after some time).

Logs
Doesn't apply, no explicit errors present.

Configuration

  1. OpenVINO Model Server 2023.1.d789fb785, OpenVINO backend 2023.1.0.12185.9e6b00e51cd
  2. Checked on 12th Gen Intel(R) Core(TM) i9-12900H and on AMD Ryzen 9 5950X 16-Core Processor
  3. Config and models included in the archive

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.