lance-format / lance-format/lance
`load_indices` should include index config
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
Ideally the output from load_indices would allow to both know
- Whether the index is an IvfPq index or a Btree index
- What parameters were used to train the index
Today I can kind of guess the first one based on the type of column but once we add more vector index types this will no longer be possible.
I have no way today of getting the parameters. This can be very useful because users may forget these things and want to examine them (e.g. because they've learned more about vector indices and now they want to know if they need to rebuild their index or not).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the load_indices entry point and tracing how index metadata is currently returned. Determine where the IvfPq or Btree type and training parameters are stored or exposed; done means load_indices reports both the index configuration and its parameters, with coverage for the supported index types.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100