zilliztech / zilliztech/VectorDBBench
Cannot run vectordb-bench in offline env
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 438
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 1
Description
Meet some issues when run vectordb-bench in offline env.
Reproduce steps:
- download cohere_medium_1m dataset
- run milvus standalone
- install vectordb-bench official pip package
pip install vectordb-bench
pip install vectordb-bench milvus
- run "init_bench", and then run task, flow will error out with this traceback:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/caiyd/miniconda3/envs/vdb/lib/python3.11/site-packages/vectordb_bench/interface.py", line 182, in _async_task_v2
case_res.metrics = runner.run(drop_old)
^^^^^^^^^^^^^^^^^^^^
File "/home/caiyd/miniconda3/envs/vdb/lib/python3.11/site-packages/vectordb_bench/backend/task_runner.py", line 120, in run
self._pre_run(drop_old)
File "/home/caiyd/miniconda3/envs/vdb/lib/python3.11/site-packages/vectordb_bench/backend/task_runner.py", line 112, in _pre_run
self.ca.dataset.prepare(self.dataset_source, filters=self.ca.filters)
File "/home/caiyd/miniconda3/envs/vdb/lib/python3.11/site-packages/vectordb_bench/backend/dataset.py", line 368, in prepare
source.reader().read(
File "/home/caiyd/miniconda3/envs/vdb/lib/python3.11/site-packages/vectordb_bench/backend/data_source.py", line 133, in read
if (not local_file.exists()) or (not self.validate_file(remote_file, local_file)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/caiyd/miniconda3/envs/vdb/lib/python3.11/site-packages/vectordb_bench/backend/data_source.py", line 149, in validate_file
info = self.fs.info(remote)
^^^^^^^^^^^^^^^^^^^^
File "/home/caiyd/miniconda3/envs/vdb/lib/python3.11/site-packages/fsspec/asyn.py", line 118, in wrapper
return sync(self.loop, func, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- want to comment out this line "if (not local_file.exists()) or (not self.validate_file(remote_file, local_file)):",
so I uninstall vectordb-bench, and install it locally, but it cannot start up
(vdb) ~/work/zilliz/VectorDBBench/ [tags/v1.0.10] pip uninstall -y vectordb-bench
Found existing installation: vectordb-bench 1.0.11
Uninstalling vectordb-bench-1.0.11:
Successfully uninstalled vectordb-bench-1.0.11
(vdb) ~/work/zilliz/VectorDBBench/ [tags/v1.0.10]
(vdb) ~/work/zilliz/VectorDBBench/ [tags/v1.0.10] pip cache purge
Files removed: 286 (16.3 MB)
(vdb) ~/work/zilliz/VectorDBBench/ [tags/v1.0.10]
(vdb) ~/work/zilliz/VectorDBBench/ [tags/v1.0.10] pip install -e ".[milvus]"
Obtaining file:///home/caiyd/work/zilliz/VectorDBBench
Installing build dependencies ... done
......
55130281d18c9ffcdf77cae9
Stored in directory: /tmp/pip-ephem-wheel-cache-n7c1wn_2/wheels/ac/03/a2/762233e655726a69db9d9b05989bd48a40b5ae020618dc4590
Successfully built vectordb-bench
Installing collected packages: vectordb-bench
Successfully installed vectordb-bench-1.0.10
(vdb) ~/work/zilliz/VectorDBBench/ [tags/v1.0.10]
(vdb) ~/work/zilliz/VectorDBBench/ [tags/v1.0.10] pip list | grep vectordb-bench
130:vectordb-bench 1.0.10 /home/caiyd/work/zilliz/VectorDBBench
(vdb) ~/work/zilliz/VectorDBBench/ [tags/v1.0.10]
(vdb) ~/work/zilliz/VectorDBBench/ [tags/v1.0.10] init_bench
Traceback (most recent call last):
File "/home/caiyd/miniconda3/envs/vdb/bin/init_bench", line 3, in <module>
from vectordb_bench.__main__ import main
File "/home/caiyd/work/zilliz/VectorDBBench/vectordb_bench/__main__.py", line 6, in <module>
from . import config
ImportError: cannot import name 'config' from 'vectordb_bench' (unknown location)
(vdb) ~/work/zilliz/VectorDBBench/ [tags/v1.0.10]
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with vectordb_bench/backend/data_source.py, especially read() and validate_file(), and reproduce the offline dataset flow through init_bench. Also inspect vectordb_bench/main.py and the package's config import for the editable-install failure. Done should mean the benchmark can use the downloaded dataset offline and init_bench starts from a local installation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100