badoo / badoo/pinba2

Idea: improve select-s performance with lockless word_id -> word translations

Open
#23 1 comment 0 reactions 1 assignee Claimed by @anton-povarov View on GitHub
enhancement performance improvement
Dominant language
C++
Stars
135
Forks
17
PR merge metrics
No merged PRs in 30d

Description

Currently to translate word_id -> word_str (done for each key in each selected row, potentially millions of times per select) - we need to read lock global dictionary shard.

This incurs significant overhead just for locks themselves.
An in cases where contention might be high (when per-repacker caching is inefficient, e.g. nginx urls) - might also slow down new packets processing.

The lock is only needed, because we use `std::deque` to find a word by offset, and it might get inserted into while we're reading (changing it's structure).

**The proposed idea** is to rework dictionary shard to use just a contiguous `mmap()`-ed memory region, enabling fully-lockless read-at-offset path (as the `mmap()`-ed region pointer never changes).

The `word_t` size is less than modern x86 CPU's cache line size, but due to the strong x86 cache-coherence model - this might only incur a performance penalty, but not compromise correctness.

As far as i understand it, anyway :)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.