dragonflydb / dragonflydb/dragonfly
Perf: Introduce CoW for Large String Reads (GET/MGET)
- Dominant language
- C++
- Stars
- 31.5k
- Forks
- 1.3k
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 127
Description
## Problem
When serving a large string value (e.g. via `GET` or `MGET`), Dragonfly always copies the value
from the shard thread before handing it off to the IO thread. For large blobs this means a heap
allocation plus a full `memcpy` on every read request — even when the value hasn't changed. Under
high-QPS read-only traffic with large values this becomes a significant source of allocator pressure
and memory bandwidth consumption.
A simple reproduction: populate Dragonfly with 1 MB string values and run `dfly_bench` in read-only
mode. The overhead will be clearly visible (TBD: to establish the baseline).
## Proposed Solution: Ref-counted CoW on the Read Path
### Background
All non-inline Dragonfly objects are wrapped in `detail::RobjWrapper`, which contains a raw pointer
and metadata. This gives us a natural place to track in-flight reads without changing the external
API.
### Read path (zero-copy)
1. When a large string read (`OBJ_STRING`, value > threshold) is dispatched asynchronously, mark
the underlying object as "being read" — either via pointer tagging on the internal pointer or via
a metadata bit in `RobjWrapper`.
2. Register the internal pointer in a **pending-read map** keyed by pointer address, with an
associated reference counter. Each concurrent reader increments the counter.
3. Pass the raw pointer (no copy) to the IO thread for serialization.
4. When the IO thread finishes serializing, it decrements the refcount. When the count reaches zero
it dispatches a lambda back to the shard to clear the flag (or free the orphaned object — see
write path below).
### Write path (CoW)
When a write arrives for a key whose current value has the pending-read flag set:
1. Allocate a new object for the incoming write.
2. Immediately update the `DbSlice` entry to point to the new object — the table stays consistent.
3. Move the old object aside (detach it from the table). It is now owned exclusively by the
pending-read map and will be freed when its refcount drops to zero.
This means writers are never blocked; readers always see a valid, stable pointer for the duration
of their IO.
### Pending-read map placement
Two options, open for discussion:
- **`EngineShard`** — natural fit since the map is keyed by pointer address and the shard already
owns the object lifetime.
- **`DbSlice`** — may be simpler to integrate with the write-path check since writes go through
`DbSlice`.
### Size threshold
Apply this optimization only for `OBJ_STRING` values larger than a hardcoded constant (initially
**1 KB**). We do not want to handle the complexities around SmallString, so it should be higher than
its maximum length. Below that threshold the existing copy path is cheaper than the refcount bookkeeping.
The constant should be tuned based on the benchmarks.
## Scope
- Initial implementation: `GET` and `MGET`.
- Other commands/types that read large blobs can be added incrementally.
## Expected Impact
For read-intensive workloads with large string values the gain should be substantial:
- Eliminates one heap allocation and one `memcpy` per read.
- Reduces allocator pressure under high QPS.
- Reduces memory bandwidth, especially relevant for multi-core IO thread fan-out.
Synthetic benchmark: populate with 1 MB values, run `dfly_bench` read-only, compare throughput and
latency percentiles before/after.
## Open Questions
- Pointer tagging vs. metadata bit — which is cleaner given the current `RobjWrapper` layout?
- `EngineShard` vs. `DbSlice` for the pending-read map.
- Whether the threshold (1 KB) should be a runtime flag or stay as a compile-time constant until
benchmarks settle.
- Impact on snapshotting / replication?
Contributor guide
Assessment
This issue has not been assessed yet.