apple / apple/foundationdb

Reduce arena allocations when materializing RocksDB range read results

Open
#13,272 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

## Description

`KeyValueRef` deep-copy construction currently performs two independent arena allocations for every returned key-value pair during RocksDB range iteration.

This occurs in the storage-server range read path, where every returned RocksDB key/value slice is materialized into a `RangeResultRef` using `push_back_deep` / `emplace_back_deep`.

For large range scans, this introduces unnecessary allocator overhead and additional allocation metadata traffic in the hot read path.

## Root Cause

File paths:

```text
fdbserver/kvstore/KeyValueStoreRocksDB.actor.cpp
fdbserver/kvstore/KeyValueStoreShardedRocksDB.actor.cpp
fdbclient/include/fdbclient/FDBTypes.h
```

Current range materialization path:

```cpp
KeyValueRef kv(toStringRef(cursor->key()), toStringRef(cursor->value()));
result.push_back_deep(result.arena(), kv);
```

`push_back_deep` constructs:

```cpp
KeyValueRef(Arena& a, const KeyValueRef& copyFrom)
: key(a, copyFrom.key), value(a, copyFrom.value) {}
```

which in turn performs two separate `StringRef(Arena&, ...)` allocations:

```cpp
StringRef(Arena& p, const StringRef& toCopy)
: data(new(p) uint8_t[toCopy.size()]), length(toCopy.size()) {
memcpy((void*)data, toCopy.data, length);
}
```

As a result, every returned KV pair performs:

* one allocation for the key payload,
* one allocation for the value payload,

even though both payloads are always materialized together as part of the same `KeyValueRef`.

## Proposed Solution

Reduce allocation overhead during range-result materialization by storing key and value payload bytes more efficiently during `KeyValueRef` deep-copy construction, while preserving existing:

* `StringRef` behavior,
* `RangeResultRef` layout,
* RocksDB iterator lifetime guarantees.

Contributor guide

Open the contributing guide

Research direction

Read fdbclient/include/fdbclient/FDBTypes.h alongside the range materialization paths in fdbserver/kvstore/KeyValueStoreRocksDB.actor.cpp and KeyValueStoreShardedRocksDB.actor.cpp. Trace KeyValueRef deep-copy construction through push_back_deep and emplace_back_deep, then verify that range results preserve StringRef behavior, RangeResultRef layout, and iterator lifetime guarantees while reducing per-pair allocation overhead.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, databases
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.