hazelcast / hazelcast/hazelcast-cpp-client

[TRACKING ISSUE] SQL: New row serialization protocol [API-2366]

Open
#800 1 comment 0 reactions 0 assignees View on GitHub
to-jira
Dominant language
C++
Stars
91
Forks
54
Avg merge
1d 12m
Merged PRs (30d)
5

Description

The tracking issue for the Java side PR.

**SQL: New row serialization protocol**

See https://github.com/hazelcast/hazelcast/pull/18089 for details.

---

This PR introduces the new serialization mechanics for SQL row data.

Before the PR, we serialized every value as `Data`. It had two problems:
1. It was inefficient.
2. It serialized some types (e.g. decimal) in a way, that the non-Java client cannot read.

With the new PR, we introduce the new custom codec `SqlPageCodec` that serializes values using a custom built-in list serializer depending on the type of the column. The main points:
1. We define the `SqlPage` class that may store values either in the row format or in the columnar format. We use the row format on the member size, and then convert it to the columnar format during decoding. This allows us to avoid copying between row format and columnar format on the member-side, making encoding almost garbage-free.
2. For every column, we pick the proper encoder based on the column type. Since every column in SQL could be nullable, we introduce several new list encoders for fixed-size types that encode nulls in the form of bitmap. See `ListCNFixedSizeCodec` and classes that use it.
3. We support custom encoders only for `VARCHAR` , `BOOLEAN`, `TINYINT`, `INTEGER`, and `BIGINT` types. Other types are still serialized using the `Data` object. We will add more custom encoders gradually. In the end, only the `OBJECT` type will be serialized as `Data`. The idea is to merge the infrastructure ASAP, and then focus on specific types (which could be time-consuming, because we need to align the encoding format with the new serialization format). This should be done before the 4.2 release.
4. The test `SqlPageCodecTest` tests all combinations of null and non-null values for every type.

Protocol part: https://github.com/hazelcast/hazelcast-client-protocol/pull/370
Supercedes https://github.com/hazelcast/hazelcast/pull/18018

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.