dolthub / dolthub/doltlab-issues

Add Garbage Collection on the remote store.

Open
#63 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
10
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Customer using DoltLab requests garbage collection for remote data. The goal of this would be to potentially improve performance (decrease latency) of DoltLab for certain use cases. Related to this [issue](https://github.com/dolthub/doltlab-issues/issues/61).

Currently DoltLab/DoltHub does not remove any stale or unused table-files from remote storage, and it's possible that this negatively impacts performance of each application.

A possible near-at-hand solution here would be to add a manual button to DoltHub/DoltLab that allows database admins the ability to trigger a "remote store garbage collection" Job. If admins notice performance issues on a DoltLab instance (or specific database) the Job would conjoin the table-files of the remote store into a single, large table-file containing all the database data and remove the superfluous table-files.

Implementing such a button on DoltHub is more complicated as it would require changes to the billing pipeline.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.