tarantool / tarantool/tarantool

Investigate if disabled tuple compression still affects memtx performance

Open
#6,964 0 comments 0 reactions 1 assignee View on GitHub

@CuriousGeorgiy is already working on this.

Since Apr 8, 2022.

performance
Dominant language
Lua
Stars
3.7k
Forks
419
Avg merge
1d 23h
Merged PRs (30d)
88

Description

Tuple compression (https://github.com/tarantool/tarantool/pull/6721) added two new methods to the index and iterator vtabs:

https://github.com/tarantool/tarantool/blob/cc768ea82cda625f97bb1e7d20fd7433514e0ec8/src/box/index.h#L429-L430

https://github.com/tarantool/tarantool/blob/cc768ea82cda625f97bb1e7d20fd7433514e0ec8/src/box/index.h#L240-L245

The new methods are used by memtx to get a tuple stored in an index as is, i.e. without decompression. The get and next method are implemented as a wrappers around the "raw" methods:

https://github.com/tarantool/tarantool/blob/cc768ea82cda625f97bb1e7d20fd7433514e0ec8/src/box/memtx_engine.cc#L1670-L1685

To avoid a possible performance penalty for indexes that don't enable tuple compression, we initially added a separate vtab for compressed indexes. The uncompressed index vtab would use the "raw" methods directly while the compressed index vtab would use the above-mentioned wrappers. However, this trick proved to be buggy and quite difficult to fix and maintain so we eventually decided to drop it (https://github.com/tarantool/tarantool/pull/6962).

Need to:

  1. Investigate if the optimization we used is worth resurrecting. (Maybe the performance penalty can be neglected at all?)
  2. If the optimization is worthwhile, fix the bugs and resurrect it.

Ideas on how to fix the optimization:

  • Call get_raw from get directly (without function pointer), inline memtx_prepare_result_tuple, and move is_compressed flag to struct tuple for better cache locality.
  • Rebuild all tuples on space alter and use a different vtab for compressed indexes.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.