tarantool / tarantool/doc

New Apache Arrow boolean interface

Open
#5,728 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

c api memcs
Dominant language
CSS
Stars
15
Forks
49
Avg merge
1d 13h
Merged PRs (30d)
3

Description

Since: Tarantool EE 3.9

Now Tarantool uses bit-packed booleans in Apache Arrow interfaces
(box_insert_arrow, box_index_arrow_stream and
box_raw_read_view_arrow_stream). Affected engines are Quiver and MemCS.

Note that there is no sense in using boolean in MemCS since they are stored
in byte-per-value layout under the hood and add overhead for
translation to bit-per-value layout when reading or inserting them with
Arrow. (This is temporary - sooner or later we will support storage of
bit-wise booleans).

Quiver on-disk boolean data from old versions is incompatible with
Tarantool 3.9.0, so it's impossible to work with old on-disk booleans
on Tarantool 3.9.0 and newer.

For the sake of backward compatibility, byte-packed booleans are still
supported in box_insert_arrow for MemCS - it will be removed
in Tarantool 4.0.
Requested by @drewdzzz in https://github.com/tarantool/tarantool-ee/commit/0b2a8eec1a894f80f8fd9426d198df752250a6a4.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the documentation for box_insert_arrow, box_index_arrow_stream, and box_raw_read_view_arrow_stream. Document the bit-packed boolean behavior, the affected Quiver and MemCS engines, the Quiver on-disk compatibility limitation since Tarantool 3.9.0, and the temporary byte-packed MemCS compatibility path through Tarantool 4.0.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.