lancedb / lancedb/lancedb

Why does the memory usage grow so quickly on the gist1m-960 dataset?

Open
#2,454 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
Rust
Stars
11.4k
Forks
1.1k
Avg merge
1d 15h
Merged PRs (30d)
140

Description

Why does the memory usage grow so quickly on the gist1m-960 dataset? Is it related to the distribution of the data?

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the memory growth on the gist1m-960 dataset and comparing it with the dataset's distribution. The issue mentions no files, tests, or entry points, so trace the relevant memory behavior from the reproduction. Done means determining whether the distribution explains the growth and documenting the cause.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.