qdrant / qdrant/vector-db-benchmark

`max_optimization_threads: 0` doesn't disable indexing

Open
#66 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
371
Forks
155
Avg merge
1d 22h
Merged PRs (30d)
4

Description

While running vector-db-benchmarks I've noticed that

  • we are updating the collection with max_optimization_threads: 0 before uploading points
  • and then once again with max_optimization_threads: 1 after upload is finished
  • I assume this is done to disable optimization/indexing during points upload
  • but I've noticed that max_optimization_threads: 0 doesn't disable indexing
    • I've disabled the max_optimization_threads: 1 request and had a collection fully indexed with max_optimization_threads: 0

If max_optimization_threads was intended to disable/enable indexing, then vector-db-benchmark can be updated to use indexing_threshold: 0 instead.

Screenshot

Notice indexed_vectors_count near the top and max_optimization_threads at the bottom.

Screenshot 2023-10-18 at 16 13 13

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading engine/clients/qdrant/configure.py lines 41-46 and engine/clients/qdrant/upload.py lines 35-40 to trace both collection updates. Compare the observed behavior with the linked bulk-upload guidance for indexing_threshold: 0. Done means the benchmark's upload configuration reliably prevents indexing, or the issue's intended semantics are clarified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.