dragonflydb / dragonflydb/dragonfly

Design: Memory efficient OAH Entry Encoding for OAHSet

Open
#7,652 12 comments 0 reactions 1 assignee Claimed by @BorysTheDev View on GitHub
Dominant language
C++
Stars
31.5k
Forks
1.3k
Avg merge
1d 10h
Merged PRs (30d)
127

Description

## Context

`OAHEntry` (`src/core/oah_entry.h`, `oah_entry.cc`) is the entry type behind `OAHSet`
(`src/core/oah_set.h`), the open-addressing hash set that can back Redis SETs
(`kEncodingStrMap2`, gated by `--use_oah_set`). Each entry is a single `uint64_t`
`data_` whose meaning today is **implicit and scattered across pointer tag bits**:

- bit 0 `kVectorBit` — slot holds a `PtrVector` collision vector
- bit 1 `kExpiryBit` — entry has a 4-byte TTL
- bit 2 `kSsoBit` — key length field is 1 byte instead of 4
- bits 52–63 — 12-bit ext-hash, read by the SIMD probe *without* dereferencing
- the masked pointer points to a heap buffer laid out as `[expiry?][key_size][key]`

This works but is rigid: every new layout variant costs a scarce pointer tag bit,
the key is always stored verbatim (no compression), and the format cannot describe a
*value* — so `OAHSet` is permanently set-only.

**Goal of this change:** make the entry **self-describing** by moving the
*content-encoding* metadata out of pointer tags and into a **descriptor byte at the
front of the heap buffer**. In its trivial form the descriptor just selects a 1-byte
vs 4-byte key length; it is also the hook for **ASCII bit-packing** of keys (8→7
bytes, reusing the CompactObj machinery). The descriptor reserves an orthogonal field
for a *value* encoding (pointer-to-string / embedded string / double / int) so that a
later effort can grow `OAHSet` into a map backing HASH and ZSET.

**Scope of this document:** the **keys-only set case** (descriptor byte + ASCII key
encoding). Value encoding is covered under *Hash-map support (further steps)*, not an
implementation target here.

**Design decisions locked in:**
1. Scope = **keys-only now**; values are deferred to *Hash-map support (further steps)*.
2. **TTL/expiry presence stays a pointer tag** (`kExpiryBit`), *not* in the descriptor.
`HasExpiry()` must remain a free register bit-test on the iteration hot path; moving
it into the buffer would force a cache-line fetch per entry even on no-TTL sets.

## Why OAHSet, over `dense_set.h`

`DenseSet` (`src/core/dense_set.h`, the base for `StringSet`/`StringMap`/`ScoreMap`) and
`OAHSet` solve the same problem — a memory-dense, TTL-aware hash table — but make
different structural choices. OAHSet favors contiguous arrays, tagged words, and SIMD
probing over the pointer-chased linked chains DenseSet uses for collisions.

**Arrays instead of linked-list collision chains.** `DenseSet` resolves collisions by
chaining: each colliding key past the first is a separately heap-allocated `DenseLinkKey`
node (`struct DenseLinkKey : DensePtr { DensePtr next; }`, two words), linked via the
`kLinkBit`-tagged `next` pointer. Walking a chain is a sequence of dependent loads —
each `Next()` is a cache miss the prefetcher can't hide, and every link node is allocator
overhead. `OAHSet` instead lays its buckets in one flat `std::vector` and
resolves collisions by **open addressing within a 32-slot displacement window**
(`kDisplacementSize = 32`) of physically adjacent 8-byte entries. A probe is a linear
sweep over contiguous memory — sequential, prefetch-friendly, and allocation-free. Only
when a window genuinely overflows does it spill into a `PtrVector` at the
window's extension point — and that overflow is itself a **single contiguous array**, not
a chain, growing by small even increments. The result: typical lookups touch one or two
cache lines of the bucket array instead of hopping across scattered link nodes.

**Heavy pointer tagging — including a cached hash fragment.** Both tables tag the spare
high/low bits of their 8-byte slot words (both note that only ~48–52 address bits are
used, leaving 12 high bits free). `DenseSet`'s tags are purely *structural* — `kLinkBit`,
`kDisplaceBit`, `kDisplaceDirectionBit`, `kTtlBit` — so to decide whether a slot matches a
query it must still **dereference** (`GetObject()`) and compare the key. `OAHSet` goes
further: it packs a **12-bit extended hash** of the key into bits 52–63 of the entry word
itself (`kExtHashShiftedMask`). A probe compares this fragment against the query's
ext-hash *directly on the bucket word* and rejects non-matches **without ever touching the
heap buffer**. The expensive key dereference/`memcmp` happens only on the rare fragment
collision. Tags also fold TTL presence (`kExpiryBit`) and the small-string length width
(`kSsoBit`, generalized by this design into the descriptor) into the same word, avoiding
extra indirections and per-entry metadata allocations.

**SIMD probing.** Because `OAHPtr` is exactly 8 bytes
(`static_assert(sizeof(OAHPtr) == sizeof(uint64_t))`), **four** slots fill one 32-byte
AVX2 register. `ProbeLanes`/`ProbeWindow` (`oah_set.cc`) load four buckets at once, extract
all four ext-hash fragments in parallel, compare them against the target, and emit a
*candidate* bitmask plus an *empties* bitmask in a handful of instructions. The whole
32-slot window is cleared in 8 such 4-lane steps; the overflow vector is probed 2 lanes at
a time (SSE), which is why vector sizes are kept even. `DenseSet` has no vectorization — it
inspects one `DensePtr` and follows one chain link at a time. The same SIMD word-scan also
powers empty-slot search (`FindEmptyAround`) and random-member selection (`ScanRange`).

**Summary.** For the common no-collision / shallow-collision case, OAHSet replaces a
pointer chase + dereference + full key compare with a vectorized scan of a contiguous
array that compares cached hash fragments and dereferences the heap buffer only on a true
candidate. The trade-off versus DenseSet's chaining is tighter memory locality, fewer
allocations, and use of SIMD — the context in which the self-describing entry encoding
below is worth adding.

## Type model: `OAHPtr` and `OAHDescriptor`

Today the single type `OAHEntry` conflates two roles: it is both the **8-byte tagged
slot** (the `uint64_t data_` stored in the bucket array and the collision vector, owning
the allocation) and, conceptually, the **record it points at**. That was harmless while
the record was an anonymous `char*` buffer, but this design gives the region real structure
(descriptor + key + optional value), so the two roles are split into two types:

- **`OAHPtr`** — the 8-byte tagged pointer. Owns the pointee; holds the structural tags
(`kVectorBit`, `kExpiryBit`, ext-hash); this is what the SIMD probe loads four-at-a-time.
An `OAHPtr` points to **either** a single record **or**, when `kVectorBit` is set, a
`PtrVector` collision array. Structural / ownership operations live here:
`IsVector`/`AsVector`, `GetHash`/`SetExtHash`, `HasExpiry`, `Insert`, `Clear`,
`AllocSize`.
- **`OAHDescriptor`** — *not a stored object*. It is the **logical representation of the
untyped memory region** that an `OAHPtr` points to: a non-owning lens that interprets the
raw `zmalloc` bytes `[descriptor byte][expiry?][keylen][key][value?]` according to the
leading descriptor byte. There is no C++ type in the allocation — the "structure" exists
only in how `OAHDescriptor`'s methods compute offsets. Content accessors live here:
`KeyEncoding`, `GetKeySize`, `KeyEquals`, `GetKey`, `KeyIfRaw`, `KeyHash`, and (future)
value getters. The type is named after the leading **descriptor byte** it decodes.

This mirrors `dense_set.h`: `DensePtr` is the tagged 8-byte slot that points to an object or
a `DenseLinkKey`; `OAHPtr` is the direct analogue, and `PtrVector` is the contiguous
collision array (cf. DenseSet's chained `DenseLinkKey` nodes).

**The region pointer.** `OAHDescriptor`'s `char* p_` is exactly `OAHPtr::Raw()` —
`data_ & ~kTagMask` (low 3 bits and high 12 bits stripped). It is only a valid record
pointer when `!IsVector()`; with `kVectorBit` set the same masked address is a
`PtrVector` instead, so the tag selects which lens applies.

**Expiry coupling.** Key/value offsets within the region depend on whether the 4-byte
expiry field is present, but expiry presence is an `OAHPtr` tag (kept off the descriptor
byte so `HasExpiry()` stays a no-deref register test, per the locked decisions). The region
is therefore *not* fully decodable from `p_` alone. Resolution: `OAHPtr::AsDescriptor()`
mints a lightweight `OAHDescriptor` carrying `p_` **plus the expiry flag copied from the
tag** — single source of truth preserved, the lens still knows its layout. (Had we put
expiry-presence in the descriptor byte, `OAHDescriptor` would be a pure function of `p_`,
at the cost of a dereferencing `HasExpiry()`.)

```cpp
class OAHPtr { // 8 bytes; owns the allocation
uint64_t data_;
char* Raw() const; // data_ & ~kTagMask
OAHDescriptor AsDescriptor() const;// { Raw(), HasExpiry() }; valid iff !IsVector()
PtrVector& AsVector();
};
class OAHDescriptor { // logical view over the untyped region; not stored
char* p_; // == OAHPtr::Raw()
bool has_expiry_; // copied from the OAHPtr tag
bool KeyEquals(std::string_view) const; void GetKey(std::string*) const; /* ... */
};
```

In the sections below, the 8-byte slot type is `OAHPtr` and the logical record view is
`OAHDescriptor`; the **descriptor byte** always means the leading metadata byte of the
region.

## Table layout

At the top level `OAHSet` is a single flat array of slots —
`Buckets = std::vector` (`entries_` in `oah_set.h`). There are no separate bucket
objects: a "bucket" is a logical window of `kDisplacementSize` (= 32) physically adjacent
`OAHPtr` slots within that vector. A key hashes to a window start and takes the first empty
slot in its 32-slot window (open addressing); the SIMD probe scans the window four slots at
a time.

Each `OAHPtr` slot is in one of three states:

- **empty** — `data_ == 0`.
- **single record** — points to one heap record (descriptor + key [+ value]), decoded via
`OAHDescriptor`.
- **overflow array** — `kVectorBit` set; points to a contiguous `OAHPtr[]` (a
`PtrVector`) holding the entries that collided past the window. This is the
collision list, and it is a **packed array, not a linked list**: one allocation of N
`OAHPtr` slots (capacity starts at 2 and grows in steps of 2), each pointing to its own
heap record — versus DenseSet, which links one separately-allocated `DenseLinkKey`
node per collision. The array hangs off the window's extension-point slot
(`bid | (kDisplacementSize - 1)`, the window's last slot).

So the structure is two levels of `OAHPtr` arrays — the top-level bucket vector and the
per-window overflow arrays — and every non-empty `OAHPtr` in either level points to a
heap record:

```
entries_ (std::vector, one contiguous array; each cell = one 8-byte OAHPtr)
┌─────┬─────┬─────┬─ … ─┬─────┐
│ p0 │ 0 │ p2 │ │ pX │
└──┬──┴─────┴─────┴─────┴──┬──┘
│ single record │ kVectorBit set → overflow array
▼ ▼
record OAHPtr[] (contiguous, sized to the collision list)
[desc│key│…] ┌─────┬─────┬─────┐
│ q0 │ q1 │ q2 │ each → its own record
└──┬──┴──┬──┴──┬──┘
▼ ▼ ▼
record record record
```

## Invariants that must not change

- `static_assert(sizeof(OAHPtr) == sizeof(uint64_t))` and `alignof == 8`
(`oah_set.h:156-157`). The AVX2 4-lane probe (`EntryWide`) depends on it — the 8-byte
constraint is on the *slot*, never the record. `OAHPtr::data_` is not growing — only the
*heap record* layout changes.
- The SIMD probe (`ProbeLanes`/`ProbeWindow`, `oah_set.cc`) reads only
`kExtHashShiftedMask` and `data_ == 0`. It must keep working with **zero
dereferences**, so the **12-bit ext-hash and `kVectorBit` stay in the pointer**.
- Empty detection keys on `data_ == 0`; the descriptor lives in the buffer, so empty
slots are still all-zero. Safe.

## What moves where

| Metadata | Today | After |
|---|---|---|
| `kVectorBit` (bit 0) | pointer tag | **stays** pointer tag (needed pre-deref) |
| ext-hash (bits 52–63) | pointer tag | **stays** pointer tag (SIMD probe) |
| `kExpiryBit` (bit 1) | pointer tag | **stays** pointer tag (hot `HasExpiry()`) |
| `kSsoBit` (bit 2) | pointer tag | **retired** → length lives in the descriptor byte |
| key length | separate 1B/4B field | descriptor byte (7-bit inline, escape for ≥127) |
| key byte encoding | always raw | descriptor byte 1-bit encoding (RAW / ASCII) |
| value encoding | n/a | a separate value descriptor byte (maps; future) |

Bit 2 is freed in the pointer; bit 1 is retained. The descriptor byte becomes the single
source of truth for **content encoding** *and* length; the pointer remains the source of
truth for **structure** (vector / empty / ext-hash) and **expiry presence**.

## Layouts

### The `OAHPtr` word (8 bytes, tagged pointer)

| Bits | Field | Meaning |
|---|---|---|
| `0` | `kVectorBit` | pointee is a `PtrVector` collision array (else a single record) |
| `1` | `kExpiryBit` | record carries a 4-byte expiry field |
| `2` | *(freed)* | was `kSsoBit`; now encoded in the descriptor byte |
| `3..51` | pointer | record / vector address (`Raw() = data_ & ~kTagMask`) |
| `52..63` | ext-hash | 12-bit key-hash fragment; read by the SIMD probe without a deref |

### Descriptor byte (offset 0 of the record)

The format is **biased toward small strings**: for any key shorter than 127 bytes the
descriptor byte alone encodes the encoding *and* the full length, so a short set member
carries exactly **one** byte of metadata — no separate length field.

| Bits | Field | Values |
|---|---|---|
| `7` | KeyEncoding | `0` RAW (verbatim bytes) · `1` ASCII (7-bit packed) |
| `6:0` | KeyLength | `0..126` = the key's **logical length**, inline · `127` = escape: the length is a varint following the descriptor (see record layout) |

A single encoding bit is all RAW-vs-ASCII needs today; the design deliberately does not
reserve codes for hypothetical future encodings (INT, Huffman, …). If one is added later it
can claim a length value as a sentinel or steal the top length bit then — there is no need
to pay for it now.

The escape uses [LEB128](https://en.wikipedia.org/wiki/LEB128) (Little-Endian Base 128), a
variable-length integer: each byte carries 7 length bits in its low bits, and the high bit
(`0x80`) is a continuation flag — set means another byte follows. Any other compact varint
would do; LEB128 is just the chosen scheme.

The escape is only reached when the length is ≥ 127, which the descriptor already
guarantees, so the varint stores **`length − 127`** rather than the full length. This
shifts its range down: a single continuation byte covers `length − 127` in `0..127`, i.e.
lengths `127..254`; two bytes cover lengths up to `127 + 16383`; and so on.

Notes:
- **No `ValueKind` / `ValueHasTtl` in the key descriptor.** Value encoding lives in its
own value descriptor byte, present only in map records (see *Hash-map support*). Per-value TTL is *not*
needed: a HASH field is itself a record, so the record's expiry (`kExpiryBit` +
expiry field) already provides HEXPIRE per-field TTL.
- **Length is the logical (decoded) length**, not the packed/allocated size — for ASCII
the packed byte count on the heap is simply `binpacked_len(KeyLength)`.

### Heap record (the region `OAHDescriptor` decodes)

The descriptor byte sits at a **fixed offset (0)** — you read it before you know the rest
of the layout. Expiry, when present, follows at the fixed offset 1 so `GetExpiry()` stays
a constant-offset read; the (rare) length-continuation varint and the key bytes come after.

| Order | Field | Size | Present when | Notes |
|---|---|---|---|---|
| 1 | descriptor byte | 1B | always | `[KeyEncoding:2][KeyLength:6]` |
| 2 | expiry | 4B | `OAHPtr` `kExpiryBit` set | absolute seconds; fixed offset 1 |
| 3 | length continuation | varint (LEB128) | `KeyLength == 127` | stores `logical length − 127` (keys ≥ 127B) |
| 4 | key bytes | variable | always | RAW: `len` bytes · ASCII: `binpacked_len(len)` bytes |
| 5 | value descriptor + payload | variable | maps only | own descriptor byte + value (see *Hash-map support*) |

So the common short-key set record is just `[descriptor][key bytes]` (plus 4B expiry only
when a TTL is set).

### ASCII encoding

ASCII packing reuses `CompactObj`'s machinery (`core/detail/bitpacking.h`): 7-bit packing
that stores 8 ASCII bytes in 7. Because the descriptor records the **logical** key length
directly, the `ASCII_DOWN`/`ASCII_UP` two-code trick CompactObj needs (to recover length
from a packed size) is **unnecessary here** — `GetKeySize()` reads the descriptor, and the
packed byte count is `binpacked_len(len)`. A key is packed only if it passes
`validate_ascii_fast` and clears a minimum-size threshold (packing tiny keys saves nothing
and costs a decode); otherwise it stays RAW.

### Worked example: a set of 16-byte hex strings

Consider a SET whose members are 16-byte hexadecimal strings (e.g. session IDs). Each
member needs a per-element heap allocation; the 8-byte bucket-array slot is the same in all
three cases below, so only the per-element region differs.

| Implementation | Region contents | Bytes requested | Allocator-rounded |
|---|---|---|---|
| `StringSet` (sds) | length + 16 payload + `\0` | ≥ 18 | **24** |
| `OAHSet` RAW | 1B descriptor/length + 16 payload | 17 | **24** |
| `OAHSet` ASCII | 1B descriptor + `binpacked_len(16)` = 14 packed | 15 | **16** |

Both current implementations land in the allocator's 24-byte size class: each needs at
least one length byte plus the 16-byte payload (17), and `StringSet` also stores a trailing
`\0` (18) — both round up to 24. Hex digits are 7-bit ASCII, so the new ASCII encoding packs
the 16 bytes into `binpacked_len(16) = 14`; with the single descriptor byte the region is
15 bytes and rounds to the **16-byte** size class. That is 24 → 16 bytes per element, a
**one-third reduction** in payload allocation, with the descriptor byte fully absorbed by
the rounding that the RAW form already paid.

## Accessor API

Today `Key()` returns a `string_view` straight into the buffer, and **every** consumer
relies on that being zero-copy. ASCII-packed bytes are not the key bytes, so packed
entries cannot return a zero-copy view. Mirror `CompactObj`:

```cpp
// Hot compare path: equality without materializing a packed key.
// RAW -> size check + memcmp
// ASCII -> size check + detail::compare_packed(bin, q.data(), q.size())
bool KeyEquals(std::string_view query) const;

// Always-correct decode into caller storage (memcpy for RAW, ascii_unpack for ASCII).
void GetKey(std::string* out) const;

// Zero-copy fast path: a view into the buffer iff KeyEncoding is RAW; nullopt if packed.
std::optional KeyIfRaw() const;

// Thin wrapper over KeyIfRaw(); DEBUG-asserts RAW. Kept for raw-only call sites.
std::string_view Key() const;

KeyEnc KeyEncoding() const; // descriptor bit[7]
uint32_t GetKeySize() const; // descriptor bits[6:0], or 127 + varint if == 127
```

`compare_packed` walks the *plaintext* query against packed bytes 7-at-a-time with
early exit — **faster** than today for packed entries, identical for RAW. (Do **not**
pack the query and memcmp: tail bytes are stored verbatim and bit boundaries shift with
length — that approach is subtly wrong. `compare_packed` is the correct primitive.)

## Concrete call-site changes (`oah_set.cc` / `oah_set.h`)

Replace zero-copy `Key()` use as follows:

- `FindMatch` key compare `e.Key() != str` (`oah_set.cc:85`) → `!e.KeyEquals(str)`
- `ProbeExtensionVector` compare `re.Key() != str` (`oah_set.cc:63`) → `!re.KeyEquals(str)`
- Rehash/affiliation hashing `Hash(entry.Key())` (`ShrinkBucket oah_set.h:371`,
`CheckBucketAffiliation :494`, `RehashEntry` debug assert `:530`) → hash a decoded key;
add an `OAHDescriptor::KeyHash()` helper that decodes once (into scratch) when ASCII.
- `Scan` callback `cb(entry.Key())` (`ScanBucket oah_set.h:396,404`) → for packed
entries decode into a thread-local scratch buffer and pass a view of that. The
`ItemCb` contract (`oah_set.h:260`) is synchronous, so the scratch lifetime is fine.
- `Fill` `it->Key()` (`oah_set.h:241`) → `GetKey`/scratch (Fill rebuilds into another set).
- Rebuild constructors `OAHPtr(Key(), ...)` (`SetExpiry oah_entry.cc:70`,
`ReallocIfNeeded oah_entry.cc:119`): for defrag, **byte-copy the existing buffer**
rather than decode+re-encode (cheaper, and the encoding/length are already settled).

## Phasing (each phase = one small, reviewable PR)

**Phase 0 — rename + accessor refactor, no format change.** Rename the 8-byte slot type
`OAHEntry` → `OAHPtr` (update `Buckets`, `PtrVector`, `oah_set.*` and the static_asserts),
and introduce a non-owning `OAHDescriptor` record view minted by `OAHPtr::AsDescriptor()`
(see *Type model*). Implement `KeyEquals`, `GetKey(string*)`, `KeyIfRaw`, `KeyHash` on the
*current* layout (on `OAHDescriptor`, forwarded from `OAHPtr` where convenient) and migrate
all call sites
above off raw `Key()`. A behavior-identical refactor that isolates the format changes in
later phases.

**Phase 1 — descriptor byte, RAW only, set-only.** Add the descriptor byte at offset 0:
`KeyEncoding = RAW` (bit[7]) with the 7-bit inline length (bits[6:0]) and the `== 127`
LEB128 escape for keys ≥ 127B. **Retire `kSsoBit`** (pointer bit 2 freed); keep `kExpiryBit`
as the pointer tag. Move expiry to buffer offset 1; update `GetKeyData`/`GetExpiry`
(`oah_entry.cc:48` → `Raw()+1`)/`SetExpiry`/`GetKeySize`/`Size`/`GetExpirySize` offsets.
`Key()`/`KeyEquals` stay trivially correct (still RAW). Re-confirm `sizeof(OAHPtr)==8`.

**Phase 2 — ASCII key encoding.** Add `KeyEncoding = ASCII`. Constructor uses
`validate_ascii_fast` + size threshold to decide, `binpacked_len(len)` to size the buffer,
and `ascii_pack`/`ascii_pack_simd2` to pack; the descriptor stores the logical length, so
**no** ASCII1/ASCII2 length-recovery code is needed. `GetKeySize` just reads the
descriptor; `KeyEquals` uses `compare_packed`; `GetKey` uses `ascii_unpack`. `Key()` now
aborts on packed entries — verify no call site slipped past Phase 0. Add roundtrip/fuzz
tests across the `binpacked_len` boundary lengths (7, 8, 14, 15, 16) and across the 7-bit
length boundary (126, 127, 128).

## Cross-cutting notes

- **Endianness:** standardize multi-byte fields (expiry, future value ptr/double/int,
length) on `absl::little_endian::Store/Load` as `string_map.cc`/`score_map.cc` do;
today's raw native-int `memcpy` (`oah_entry.cc:27,36,48`) is host-endian. The
descriptor byte is endian-neutral.
- **Accounting:** `AllocSize()` (`oah_entry.h:232`, `zmalloc_usable_size(Raw())`) feeds
`obj_alloc_used_` deltas in Erase/Expire/Shrink/ClearStep. The descriptor adds 1 byte
inside the single allocation — add/remove sides stay consistent automatically in the
keys-only phases.

## Hash-map support (further steps)

When `OAHSet` is grown into a map (HASH field→value, ZSET member→score), a **second,
value descriptor byte** follows the key bytes. It has its own split — four value kinds need
2 bits, leaving 6 for an embedded-string length with an escape, in the same spirit as the
key descriptor's small-string bias. `ValueKind` codes: `PTR` (external string pointer),
`EMBED_STR` (inline string), `DOUBLE` (ZSET score), `INT` (integer-encoded value).

Layouts of one record (`OAHPtr.Raw()` points to the key descriptor byte at offset 0;
`kdsc`/`vdsc` = key/value descriptor byte, `n` = inline length):

```
EMBED_STR — small HASH value, everything in one allocation:

OAHPtr.Raw()


┌──────┬───────────┬─────────┬──────┬─────────┐
│ kdsc │ expiry? │ key │ vdsc │ value │
│ RAW │1234567890 │ "field" │ EMB │ "bar" │
│ n=5 │ (4B) │ bytes │ n=3 │ bytes │
└──────┴───────────┴─────────┴──────┴─────────┘
1B 4B opt 5B 1B 3B

PTR — large HASH value, value held in a separate allocation the record owns:

OAHPtr.Raw()


┌──────┬───────────┬─────────┬──────┬─────────┐
│ kdsc │ expiry? │ key │ vdsc │ val ptr │
│ RAW │1234567890 │ "field" │ PTR │ ●─────┼──► external sds: hdr "large value…" \0
│ n=5 │ (4B) │ bytes │ │ (8B) │
└──────┴───────────┴─────────┴──────┴─────────┘

DOUBLE — ZSET member→score, fixed 8-byte score embedded:

OAHPtr.Raw()


┌──────┬───────────┬─────────┬────────┬────────────┐
│ kdsc │ expiry? │ member │ vdsc │ score │
│ RAW │ (none) │ "elem" │ DOUBLE │ 3.14 (8B) │
│ n=4 │ │ bytes │ │ │
└──────┴───────────┴─────────┴────────┴────────────┘
```

(For comparison, a plain SET record is just the first three cells: `kdsc`, optional
`expiry`, and the key bytes — no value descriptor.) Notes:

- `DOUBLE` (ZSET score) and `EMBED_STR` own nothing external — `Clear`/`AllocSize`/
`ReallocIfNeeded` are unchanged. Land these first.
- `PTR` (large HASH values, StringMap model) embeds an 8-byte external pointer the entry
**owns**: `Clear()` must free it, `AllocSize()` must add its
`zmalloc_usable_size`, `ReallocIfNeeded` must defrag it and rewrite the pointer, and
the single-entry rebuild path must become value-aware (the current
`OAHPtr(Key(),expiry)` rebuild would drop the value). It carries the most ownership
complexity, so land `PTR` last, porting `StringMap::ReallocIfNeeded`
(`string_map.cc:220-251`) and `ObjectAllocSize` (`:284`) logic.
- **No per-value TTL field.** HEXPIRE per-field TTL is served by the entry-level expiry
(`kExpiryBit` + expiry field) — each HASH field is its own record, so the record's
expiry *is* the field's TTL.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.