lance-format / lance-format/lance
Object store cache entries cannot be cleared
@wjones127 is already working on this.
Since Jan 8, 2026.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
The ObjectStoreRegistry::get_store function in rust/lance-io/src/object_store/providers.rs use path and storage_options as key to cache object store.
https://github.com/lance-format/lance/blob/1beb29a0676d7b1700f6741fb19b6911a4d542e2/rust/lance-io/src/object_store/providers.rs#L174-L176
And it uses a passive cache eviction method, rather than periodic active eviction. That is, object stores are only evicted from the cache when it is found to be unavailable during access.
https://github.com/lance-format/lance/blob/1beb29a0676d7b1700f6741fb19b6911a4d542e2/rust/lance-io/src/object_store/providers.rs#L187-L203
This can easily lead to invalid object stores remaining permanently in the cache, preventing the memory from being released.
A typical scenario is accessing S3 using temporary credentials that require periodic refreshing. Since the access_key_id, secret_access_key, and token are part of the storage_options, object_store cache entries using expired credentials as part of the key will never be cleaned up.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.