PipedreamHQ / PipedreamHQ/pipedream

[FEATURE] Data Store record retention policy

Open
#9,285 0 comments 1 reaction 0 assignees View on GitHub
enhancement tracked internally triaged
Dominant language
JavaScript
Stars
11.7k
Forks
5.8k
Avg merge
3d 10h
Merged PRs (30d)
102

Description

**Is your feature request related to a problem? Please describe.**
The Data Stores feature provides a useful solution for synchronizing processes across multiple workflows. In this use case, data is added to a store or modified, but doesn't need to be stored in perpetuity -- however viewing data for some period after the fact can be useful for debugging purposes. Data Stores now has a storage limit, so cleaning up these rows needs to happen eventually to avoid surpassing this limit.

**Describe the solution you'd like**
I'd like the ability to set a retention policy on a per-data store level, so that records will be automatically deleted some number of days after creation or last modification, conserving the storage quota.

**Do you have a workaround?**
Records can be deleted manually, or one could create a cronjob to clean up old entries.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.