zilliztech / zilliztech/VectorDBBench

Cohere Dataset Size documentation inconsistent and off by a factor of 10 in Places

Open Beginner friendly
#878 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.2k
Forks
438
Avg merge
1d 4h
Merged PRs (30d)
1

Description

Hello,

I noticed that there is an issue with the labeling of dataset sizes for Cohere. In some cases Small, Medium and Large are listeds as 100K, 1M and 10M, but in other places its 1M, 10M, and 100M.

EDIT: Or the cohere small 768Dim 100K dataset is missing in CaseType

vectordb_bench/backend/cases.py:37

class CaseType(Enum):
    """
    Example:
        >>> case_cls = CaseType.CapacityDim128.case_cls
        >>> assert c is not None
        >>> CaseType.CapacityDim128.case_name
        "Capacity Test (128 Dim Repeated)"
    """

    CapacityDim128 = 1
    CapacityDim960 = 2

    Performance768D100M = 3
    Performance768D10M = 4
    Performance768D1M = 5

vectordb_bench/backend/dataset.py:1094

class DatasetWithSizeType(Enum):
    CohereSmall = "Small Cohere (768dim, 100K)"
    CohereMedium = "Medium Cohere (768dim, 1M)"
    CohereLarge = "Large Cohere (768dim, 10M)"
    LAIONLarge = "Large LAION (768dim, 100M)"
    BioasqMedium = "Medium Bioasq (1024dim, 1M)"
    BioasqLarge = "Large Bioasq (1024dim, 10M)"
    OpenAISmall = "Small OpenAI (1536dim, 50K)"
    OpenAIMedium = "Medium OpenAI (1536dim, 500K)"
    OpenAILarge = "Large OpenAI (1536dim, 5M)"

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the Cohere entries in vectordb_bench/backend/dataset.py with the available Cohere-related cases in vectordb_bench/backend/cases.py. Trace where these enum values are displayed or selected for Places, then verify that the Small, Medium, and Large dataset sizes are labeled consistently and that the 768-dimension 100K case is represented if required.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.