lance-format / lance-format/lance

bug: create_index accepts index_cache_size and silently ignores it

Open Beginner friendly
#9,383 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Description

LanceDataset.create_index takes an index_cache_size parameter and documents it as "The size of the index cache in number of entries. Default value is 256." Nothing reads it. _create_index_impl never puts it into the kwargs dict it hands to self._ds.create_index, so the value is dropped in Python and never reaches Rust. create_index_uncommitted forwards the same parameter into the same dead end.

The name makes the trap worse: lance.dataset(..., index_cache_size=...) is a real knob (deprecated in favour of index_cache_size_bytes, converted at 20 MiB per entry), so a user who sets it on create_index has every reason to think they sized the cache for that index. They did not, and nothing says so.

Index building has no use for an index cache either, so the parameter cannot be honoured as documented. What is missing is saying that out loud.

Expected behavior

create_index should warn that index_cache_size is ignored, the way the LanceDataset constructor already warns about the deprecated spelling there, and the docstring should say the same instead of describing a cache size that is never applied.

Lance version

13.0.0-beta.4 (main)

Language binding

Python

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating LanceDataset.create_index, _create_index_impl, and create_index_uncommitted, then compare their handling with the LanceDataset constructor's existing warning for the deprecated spelling. Done means create_index warns that index_cache_size is ignored, and its documentation states that index building does not use an index cache; ensure the related forwarding path is covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
database
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.