zilliztech / zilliztech/VectorDBBench
Cohere Dataset Size documentation inconsistent and off by a factor of 10 in Places
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 438
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 1
Description
Hello,
I noticed that there is an issue with the labeling of dataset sizes for Cohere. In some cases Small, Medium and Large are listeds as 100K, 1M and 10M, but in other places its 1M, 10M, and 100M.
EDIT: Or the cohere small 768Dim 100K dataset is missing in CaseType
vectordb_bench/backend/cases.py:37
class CaseType(Enum):
"""
Example:
>>> case_cls = CaseType.CapacityDim128.case_cls
>>> assert c is not None
>>> CaseType.CapacityDim128.case_name
"Capacity Test (128 Dim Repeated)"
"""
CapacityDim128 = 1
CapacityDim960 = 2
Performance768D100M = 3
Performance768D10M = 4
Performance768D1M = 5
vectordb_bench/backend/dataset.py:1094
class DatasetWithSizeType(Enum):
CohereSmall = "Small Cohere (768dim, 100K)"
CohereMedium = "Medium Cohere (768dim, 1M)"
CohereLarge = "Large Cohere (768dim, 10M)"
LAIONLarge = "Large LAION (768dim, 100M)"
BioasqMedium = "Medium Bioasq (1024dim, 1M)"
BioasqLarge = "Large Bioasq (1024dim, 10M)"
OpenAISmall = "Small OpenAI (1536dim, 50K)"
OpenAIMedium = "Medium OpenAI (1536dim, 500K)"
OpenAILarge = "Large OpenAI (1536dim, 5M)"
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing the Cohere entries in vectordb_bench/backend/dataset.py with the available Cohere-related cases in vectordb_bench/backend/cases.py. Trace where these enum values are displayed or selected for Places, then verify that the Small, Medium, and Large dataset sizes are labeled consistently and that the 768-dimension 100K case is represented if required.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 74/100