New Apache Arrow boolean interface
Nobody has claimed this yet.
- Dominant language
- CSS
- Stars
- 15
- Forks
- 49
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 3
Description
Since: Tarantool EE 3.9
Now Tarantool uses bit-packed booleans in Apache Arrow interfaces
(box_insert_arrow, box_index_arrow_stream and
box_raw_read_view_arrow_stream). Affected engines are Quiver and MemCS.
Note that there is no sense in using boolean in MemCS since they are stored
in byte-per-value layout under the hood and add overhead for
translation to bit-per-value layout when reading or inserting them with
Arrow. (This is temporary - sooner or later we will support storage of
bit-wise booleans).
Quiver on-disk boolean data from old versions is incompatible with
Tarantool 3.9.0, so it's impossible to work with old on-disk booleans
on Tarantool 3.9.0 and newer.
For the sake of backward compatibility, byte-packed booleans are still
supported in box_insert_arrow for MemCS - it will be removed
in Tarantool 4.0.
Requested by @drewdzzz in https://github.com/tarantool/tarantool-ee/commit/0b2a8eec1a894f80f8fd9426d198df752250a6a4.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the documentation for box_insert_arrow, box_index_arrow_stream, and box_raw_read_view_arrow_stream. Document the bit-packed boolean behavior, the affected Quiver and MemCS engines, the Quiver on-disk compatibility limitation since Tarantool 3.9.0, and the temporary byte-packed MemCS compatibility path through Tarantool 4.0.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100