facebook / facebook/rocksdb

rocksdbjni 5.17.2 RocksDB::get() returns incomplete value, which are just merged with RockDB::merge()

Open
#8,343 0 comments 0 reactions 0 assignees View on GitHub
java-api question up-for-grabs
Dominant language
C++
Stars
32.1k
Forks
6.9k
Avg merge
32m
Merged PRs (30d)
1

Description

### Expected behavior
I am using RocksDB java with 5.17.2 (the version used by current Apache Flink community).

The test is simple. I use `db::merge()` to append data to one key. And finally, I use `db::get()` to check all the stuff that I merged just now. I use `stringappendtest` merge operator (same as RocksDB used in Apache Flink).

I append value to a same key 65536 times, each time with 16KB data. At `get` invocation, the return byte array length ought to be 16KB * 65536 + 1 * 65535 = 1073807359.

### Actual behavior
The length of `db::get()` is at random.

If I change `rocksdbjni` version to 6.15.2, everything seems to work fine.

### Steps to reproduce the behavior
Use all default setting in `rocksdbjni`, ie., `WriteOptions`, `ReadOptions`, `DBOption.setCreateIfMissing(true)`.

Simple testing code
```
public void test(String key) throws RocksDBException {
for (int i = 0; i < mergeRound; i++) {
rnd.nextBytes(payload);
db.merge(key.getBytes(StandardCharsets.UTF_8), payload);
}
for (int i = 0; i < 30; i++) {
byte[] result = db.get(readOptions, key.getBytes(StandardCharsets.UTF_8));
System.out.println("Round " + i + "value size of: " + key + ": " + result.length + ", ought to be " +
(mergeRound * payload.length + mergeRound - 1));
}
db.delete(key.getBytes(StandardCharsets.UTF_8));
}
```

Since it is hard to upgrade `rocksdbjni`'s version in Flink, I was wondering if there is a way to solve this confusing condition in `rocksdbjni` 5.17.2 by repairing some RocksDB's options. Thanks for your sharing.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.