meilisearch / meilisearch/arroy

Measure and improve the constant numbers used when building the tree

Open
#35 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
311
Forks
17
PR merge metrics
No merged PRs in 30d

Description

We must take three parameters into account:

  1. Time to build the tree
  2. Relevancy of the searches
  3. Time to search in the tree

Fun fact: the lowest in the tree you are, the less impact a dummy plane has on the search cost.


https://github.com/meilisearch/arroy/blob/7fc603166af61e18e5257cf5859e6cf71ecbfeab/src/writer.rs#L248-L259

fn split_imbalance(left_indices_len: usize, right_indices_len: usize) -> f64 {
    let ls = left_indices_len as f64;
    let rs = right_indices_len as f64;
    let f = ls / (ls + rs + f64::EPSILON); // Avoid 0/0
    f.max(1.0 - f)
}

fn main() {
    dbg!(split_imbalance(29464, 18394));
    dbg!(split_imbalance(30000, 30000));
    dbg!(split_imbalance(30000, 1580));
}

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/writer.rs at split_imbalance and review the constants used while building the tree. Measure the three stated concerns—tree-build time, search relevance, and search time—using the provided imbalance examples as an initial reference. The issue does not define target values, benchmarks, tests, or acceptance criteria, so the work is not ready to start without further direction.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
search
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.