lancedb / lancedb/lancedb

Feature: allow cache to be shared between multiple table instances

Open
#1,336 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
11.4k
Forks
1.1k
Avg merge
1d 15h
Merged PRs (30d)
140

Description

### SDK

Python

### Description

Every `Table` has a cache for index & metadata. Right now that cache is newly created for each table instance. This can lead to memory issues. However, it should be possible for multiple table instances to share the same cache. This would make it easy, for example, to create a bunch of tables to handle concurrent queries.

Contributor guide

Open the contributing guide

Research direction

Start at the Python SDK's Table implementation and trace how its index and metadata cache is created for each instance. Determine how multiple Table instances could receive and share one cache, including its lifecycle, then add coverage showing that sharing works without duplicating the cache.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.