Reduce arena allocations when materializing RocksDB range read results
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
## Description
`KeyValueRef` deep-copy construction currently performs two independent arena allocations for every returned key-value pair during RocksDB range iteration.
This occurs in the storage-server range read path, where every returned RocksDB key/value slice is materialized into a `RangeResultRef` using `push_back_deep` / `emplace_back_deep`.
For large range scans, this introduces unnecessary allocator overhead and additional allocation metadata traffic in the hot read path.
## Root Cause
File paths:
```text
fdbserver/kvstore/KeyValueStoreRocksDB.actor.cpp
fdbserver/kvstore/KeyValueStoreShardedRocksDB.actor.cpp
fdbclient/include/fdbclient/FDBTypes.h
```
Current range materialization path:
```cpp
KeyValueRef kv(toStringRef(cursor->key()), toStringRef(cursor->value()));
result.push_back_deep(result.arena(), kv);
```
`push_back_deep` constructs:
```cpp
KeyValueRef(Arena& a, const KeyValueRef& copyFrom)
: key(a, copyFrom.key), value(a, copyFrom.value) {}
```
which in turn performs two separate `StringRef(Arena&, ...)` allocations:
```cpp
StringRef(Arena& p, const StringRef& toCopy)
: data(new(p) uint8_t[toCopy.size()]), length(toCopy.size()) {
memcpy((void*)data, toCopy.data, length);
}
```
As a result, every returned KV pair performs:
* one allocation for the key payload,
* one allocation for the value payload,
even though both payloads are always materialized together as part of the same `KeyValueRef`.
## Proposed Solution
Reduce allocation overhead during range-result materialization by storing key and value payload bytes more efficiently during `KeyValueRef` deep-copy construction, while preserving existing:
* `StringRef` behavior,
* `RangeResultRef` layout,
* RocksDB iterator lifetime guarantees.
Contributor guide
Research direction
Read fdbclient/include/fdbclient/FDBTypes.h alongside the range materialization paths in fdbserver/kvstore/KeyValueStoreRocksDB.actor.cpp and KeyValueStoreShardedRocksDB.actor.cpp. Trace KeyValueRef deep-copy construction through push_back_deep and emplace_back_deep, then verify that range results preserve StringRef behavior, RangeResultRef layout, and iterator lifetime guarantees while reducing per-pair allocation overhead.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, databases
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100