qdrant / qdrant/fastembed

[Bug]: warning about nodes not assigned to the preferred execution providers when running on GPU

Open
#467 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.2k
Forks
248
Avg merge
4d 8h
Merged PRs (30d)
4

Description

What happened?

Not really a bug, more of a warning when generating embeddings with fastembed-gpu following this docs:

I am getting these warnings:

[W:onnxruntime:, session_state.cc:1168 VerifyEachNodeIsAssignedToAnEp] 
Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.

[W:onnxruntime:, session_state.cc:1170 VerifyEachNodeIsAssignedToAnEp] 
Rerunning with verbose output on a non-minimal build will show node assignments.

When I print embedding_model_gpu.model.model.get_providers() I can see these available providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'] (cf. below for reproducible example)

Can this warning be dealt with elegantly? What is this really about? Thanks a lot

What is the expected behaviour?

No response

A minimal reproducible example

To be run with uv run scriptname.py:

# /// script
# dependencies = [
#   "fastembed-gpu",
# ]
# ///

from fastembed import TextEmbedding

embedding_model_gpu = TextEmbedding(
    model_name="BAAI/bge-small-en-v1.5", providers=["CUDAExecutionProvider"]
)
print(embedding_model_gpu.model.model.get_providers())

documents: list[str] = [
    "Demonstrating GPU acceleration in fastembed 1",
    "Demonstrating GPU acceleration in fastembed 2",
    "Demonstrating GPU acceleration in fastembed 3",
]

embedded = list(embedding_model_gpu.embed(documents))
print(embedded)

Running on CUDA Version: 12.4

What Python version are you on? e.g. python --version

Python 3.12.8

FastEmbed version

v0.5.1

What os are you seeing the problem on?

Linux

Relevant stack traces and/or logs

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided script with uv run scriptname.py and inspect the TextEmbedding provider output and warnings from embed. Determine whether the CUDAExecutionProvider warning is expected for this example or requires a project change, then document or verify the resulting behavior against the reported environment.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.