cockroachdb / cockroachdb/cockroach

explicitly protecting timestamps

Open
#100,470 3 comments 0 reactions 0 assignees View on GitHub
C-enhancement O-community T-sql-queries X-blathers-triaged
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Is your feature request related to a problem? Please describe.**

The [storage layer documentation](https://www.cockroachlabs.com/docs/stable/architecture/storage-layer.html#protected-timestamps) outlines that certain operations like imports or backups create "protected timestamps":

> Garbage collection can only run on MVCC values which are not covered by a *protected timestamp* . The protected timestamp subsystem exists to ensure the safety of operations that rely on historical data, such as (...)

I would like to create a protected timestamp manually so I can be sure the data will be available in the future, even if it grows older than `gc.ttlseconds`. [According to Andrew Werner](https://forum.cockroachlabs.com/t/explicitly-protecting-a-timestamp/5875/1) this is currently not possible.

Use case: We're operating a distributed system of multiple thousands of workers all operating on the same dataset, mostly reading from it. When the dataset changes each worker needs to operate on a consistent state at that point in time. Using protected timestamps, I'd like to give workers a API which allows them to query the entire dataset at the relevant point in time.

This works using [`AS OF SYSTEM TIME`](https://www.cockroachlabs.com/docs/stable/as-of-system-time.html#synopsis), however, I need to guarantee that I don't need any data older than `gc.ttlseconds`. We're aiming to keep the window between the oldest needed state and present time rather short (=minutes), but during outages I'd like to have some headroom.

**Describe the solution you'd like**

1. Add an API which allows regular clients to protected arbitrary timestamps, exempt from GC. Possibly by just `INSERT`ing into `system.protectedts_records`.
2. Add an API to un-protect timestamps. Possibly by `DELETE`ing.

This solves the following problems:

* Guarantees that the data is still there when I need it.
* Prevents bloat caused by overly big `gc.ttlseconds` values.

**Describe alternatives you've considered**

1. Raising `gc.ttlseconds` to high values of a week or so: This works, but is bound to cause storage or performance problems at some point. Andrew also pointed out that you are looking to keep `gc.ttlseconds` rather short, so allowing users to work around shorter values, if they need to, would support that goal.
4. Implement the "time travel" logic within my application: This is definitely possible, but I will end up writing most of my very own MVCC within the app. Since cockroach already has all the needed data and 98% of the needed interfaces, I'd love to avoid that.
5. using [row level `ttl`](https://www.cockroachlabs.com/docs/stable/row-level-ttl.html): Causes similar problems as implementing the logic myself. Just using `ttl`s is not enough, because I need the state to be consistent across multiple tables, not just one.

Please let me know, if I can provide more information or if you've got any questions.

Jira issue: CRDB-26455

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.