ClickHouse / ClickHouse/clickhouse-rs
Replace/supplement Serde serialization with bespoke traits
- Dominant language
- Rust
- Stars
- 559
- Forks
- 172
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 3
Description
The design choice of using Serde for serialization leads to sub-optimal DX as we have no control over the encoding of external types. Because the default serialization differs from the encoding ClickHouse uses for a lot of types, this means we end up needing a lot of adapters/wrappers/`#[serde(with = "...")]` attributes to make things work: [https://github.com/ClickHouse/clickhouse-rs/blob/25d940b7343ee6ae6344dcdb1454c2bfad47bfcd/examples/data_types_derive_simple.rs#L134-L170]()
Omitting these can lead to incorrect results, like in the case of `Ipv4Addr` (ClickHouse/clickhouse-rs#231), because the `RowBinary` protocol encodes IP addresses in little-endian but Serde uses big-endian.
Pretty much every other SQL client library out there uses bespoke traits for serialization so they have full control over the encoding. This includes [Diesel](), [`tokio-postgres`](), and of course, [SQLx]().
Contributor guide
Research direction
Start with examples/data_types_derive_simple.rs, especially the Serde adapters, and inspect how the RowBinary protocol currently encodes types. Define the scope of bespoke serialization traits and verify that the resulting encoding matches ClickHouse expectations, including the Ipv4Addr case described in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100