ClickHouse / ClickHouse/clickhouse-js
Add RowBinary format for SELECT operations
- Dominant language
- TypeScript
- Stars
- 331
- Forks
- 74
- PR merge metrics
- No merged PRs in 30d
Description
Work in progress: #257
If you have any feedback on the following, please don't hesitate to comment.
Streaming will be supported out of the box _by design_.
Target supported [ClickHouse data types](https://clickhouse.com/docs/en/sql-reference/data-types) with SELECT operations for the initial release; certain types are already completed in the branch:
- [x] Boolean
- [x] (U)Int8-32 as `number`
- [x] (U)Int64-256 as a native `BigInt`
- [x] Float32, Float64 as a `number`
- [x] String as a `string`
- [x] Nullable, LowCardinality
- [x] Date, Date32 as either a `JS Date`, a `string`, or UNIX seconds as a `number`/`BigInt` by default (needs the decision to be consistent with DateTime family types), with a custom mapper support.
- [ ] DateTime, DateTime64 - with a custom mapper support. DateTime could be either an ISO format `string` or `BigInt` UNIX seconds. DateTime64 should be either a `string` or UNIX nanoseconds as a `BigInt`.
- [x] Enum - needs a decision. Maybe as a string by default, with a custom mapper (index) -> T
- [ ] UUID as a `string`
- [x] FixedString(N) as a `string`
- [ ] IPv4/v6 as a `string`
- [x] Decimal as `string` by default, support custom mapper
- [x] Array(T) - should use custom mappers provided for the value type
- [ ] Map(K, V) - should use custom mappers provided for K and V types
- [ ] Tuple(T1, T2, ...) - should use custom mappers provided for T* types
- [ ] Geo types - likely, should use `[number, number]` for Point as the base for the remaining Geo types.
The types that may be included as well, depending on the format complexity:
- [ ] Variant(T1, T2, ...) - should use custom mappers provided for T* types.
- [ ] SimpleAggregateFunction - could be trivial; needs clarification.
- [ ] Nested - needs clarification. Could be tricky.
- [ ] JSON - maybe not, as it is still [experimental](https://clickhouse.com/docs/en/sql-reference/data-types/json); see also [this guide about working with JSON in ClickHouse](https://clickhouse.com/docs/en/integrations/data-formats/json) and the [Semi-structured columns RFC](https://github.com/ClickHouse/ClickHouse/issues/54864).
Related issues:
- #25 - branched out of that one
- https://github.com/ClickHouse/clickhouse-js/issues/212 - feature request, as RowBinary could provide performance benefits
- #215 - somewhat related; that one was about INSERT operations. However, RowBinary implementation allows us to implement data mappers for specific data types for both SELECT and INSERT; these types are Date, Date32, DateTime, DateTime64, and Decimal (taking Array or Map values into consideration, too), as the user might want to get Luxon/Decimal.js/etc objects right away.
Contributor guide
Assessment
This issue has not been assessed yet.