ClickHouse / ClickHouse/clickhouse-rs

Replace/supplement Serde serialization with bespoke traits

Open
#336 1 comment 1 reaction 0 assignees View on GitHub
1.0 blocker
Dominant language
Rust
Stars
559
Forks
172
Avg merge
1d 19h
Merged PRs (30d)
3

Description

The design choice of using Serde for serialization leads to sub-optimal DX as we have no control over the encoding of external types. Because the default serialization differs from the encoding ClickHouse uses for a lot of types, this means we end up needing a lot of adapters/wrappers/`#[serde(with = "...")]` attributes to make things work: [https://github.com/ClickHouse/clickhouse-rs/blob/25d940b7343ee6ae6344dcdb1454c2bfad47bfcd/examples/data_types_derive_simple.rs#L134-L170]()

Omitting these can lead to incorrect results, like in the case of `Ipv4Addr` (ClickHouse/clickhouse-rs#231), because the `RowBinary` protocol encodes IP addresses in little-endian but Serde uses big-endian.

Pretty much every other SQL client library out there uses bespoke traits for serialization so they have full control over the encoding. This includes [Diesel](), [`tokio-postgres`](), and of course, [SQLx]().

Contributor guide

Open the contributing guide

Research direction

Start with examples/data_types_derive_simple.rs, especially the Serde adapters, and inspect how the RowBinary protocol currently encodes types. Define the scope of bespoke serialization traits and verify that the resulting encoding matches ClickHouse expectations, including the Ipv4Addr case described in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.