RoaringBitmap / RoaringBitmap/roaring-rs

Hand tune array-bitset aggregates

Open
#166 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
957
Forks
119
Avg merge
3d 3h
Merged PRs (30d)
1

Description

arXiv:1709.07821 Roaring Bitmaps: Implementation of an Optimized Software Library (Section 4.1.1) (Section 3.2)

Cardinality tracking was added in #127, however we're leaving some perf on the floor.

We also find that if we use bit-manipulation instructions and hand-tuned assembly, we can roughly
double the speed at which we can change bits in a bitset, compared to compiler-generated machine
code with all optimization flags activated. It is true not only when setting bits, but also when clearing
or flipping them—in which cases we need to use the btr and btc instructions instead. Informally,
we tested various C compilers and could not find any that could perform nearly as well as hand-tuned
assembly code. This may, of course, change in the future.

CRoaring source

Planning

  1. Do we want to implement this optimization? 2x is a huge speed up, but only on x86
  2. The compiler is sufficiently smart to use bts but not sbb, perhaps we can write rust that compiles to the asm we want
  3. If not there's always the asm! macro (nightly only), or if want it in stable then we can write a small C lib for the asm and link it

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the cardinality tracking added in #127 and compare the Rust u32 borrowing_sub and core::arch::x86_64::_subborrow_u64 options against CRoaring's src/bitset_util.c. Done would require a confirmed optimization choice and evidence that it improves bitset aggregate performance without penalizing other architectures.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, rust
Domain
performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.