qdrant / qdrant/fastembed

ONNXRuntime taking up too much memory

Open
#483 6 comments 0 reactions 1 assignee View on GitHub

@hh-space-invader is already working on this.

Since Mar 2, 2025.

Dominant language
Python
Stars
3.2k
Forks
248
Avg merge
4d 8h
Merged PRs (30d)
4

Description

ONNXRuntime takes up too much memory (more like accumulating cause I believe it's not freeing up unused memory), when trying to embed large collections of data.
Am I missing something or is this a problem with the runtime itself.
Iam trying to embed about 10000 documents (Average Size - 3000 characters) using the JinaAI Colbert Model (Late Interaction Model)
GPU - Tesla T4 16 GB VRam

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.