activeloopai / activeloopai/deeplake

[BUG] Read-Only Vectorstore with GCS persistence goes stale

Open
#2,612 7 comments 0 reactions 1 assignee Claimed by @adolkhan View on GitHub
bug
Dominant language
C++
Stars
9.2k
Forks
722
PR merge metrics
No merged PRs in 30d

Description

### Severity

P0 - Critical breaking issue or missing functionality

### Current Behavior

When running the Deeplake Vectorstore with a GCS path, any changes and commits made by a separate Deeplake instance on the same GCS path does not get picked up by the already running Deeplake Vectorstore instance.

### Steps to Reproduce

1. Run a Deeplake Vectorstore with a Google cloud storage path in read-only mode
2. Run a separate Deeplake Vectorstore with the same GCS path and push some new data to the Vectorstore
3. Perform a search with the first Deeplake Vectorstore instance and see if the new data is reflected. The new data typically does not get reflected.

### Expected/Desired Behavior

A Deeplake Vectorstore with cloud persistence should periodically pick up and pull any changes made to the peristed data by another vectorstore instance.

Alternatively, provide a refresh method to trigger any Deeplake Vectorstore to refresh its data from cloud persistence.

### Python Version

_No response_

### OS

_No response_

### IDE

_No response_

### Packages

_No response_

### Additional Context

_No response_

### Possible Solution

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR (Thank you!)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.