facebook / facebook/rocksdb

[Feature Request]Timestamp API Feature Request

Open
#8,770 16 comments 4 reactions 1 assignee Claimed by @riversand963 View on GitHub
design discussion
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

@riversand963, after an rigorous review of the `timestamp series api`, I'm delighted to see that

1. Put/Del/Merge/Seek/Get/... user ddl operations all support timestamp
2. Compaction considers full_history_ts_low as the oldest timestamp to preserve
3. full_history_ts_low is persisted into manifest as a per-cf property

For user's convinence, I proposal these more APIs

1. Add a public api `SetFullHistoryTsLow` described below

```
// set the new full_history_ts_low to this cf, full_history_ts_low can not move back unless force == true.
Status SetFullHistoryTsLow(ColumnFamily* cf, const std::string& full_history_ts_low, bool force) {
}
const std::string& GetFullHistoryTsLow() const {}
```

2. Add a public api `TrimHistoryToTimestamp ` described below

```
// 1. Flush all memtables
// 2. Stop writes(perhaps stop writes should be controled by users)
// 3. Dispatch a CompactionJob, which takes ts as a new parameter
// 4. CompactionIterator processes each kv, and drops this kv if kv.timestmp > ts
// 5. ts should not be less than cf.full_history_ts_low
Status TrimHistoryToTimestamp(ColumnFamily* cf, const std::string& ts)
```

3. To better support `TrimHistoryToTimestamp `, each sst file should add `minTs` and `maxTs` to avoid doing too much useless scanning.

In addition, current implementation of DBIter::Seek with timestamp may deadlock, when a key has more timestamps than the `max_seek` internal variable. I will investigate more, post an issue when I've confirmed this.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.