cockroachdb / cockroachdb/cockroach

sql: support parquet logical type TIMESTAMP in EXPORT INTO PARQUET

Open
#94,678 5 comments 0 reactions 0 assignees View on GitHub
A-cdc C-enhancement E-quick-win E-starter O-community T-cdc X-blathers-triaged
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Is your feature request related to a problem? Please describe.**
Currently we are trying to export database snapshot as parquet with `EXPORT` query. We found that, as mentioned in the [CochroachDB Docs](https://www.cockroachlabs.com/docs/stable/export.html#parquet-types), CockroachDB Type `TIMESTAMP`/`TIMESTAMPTZ` is exported as Parquet Logical Type `STRING` even though Parquet Logical Type support `TIMESTAMP`. Consequently, reading from exported parquet files and casting such columns from `STRING` to `TIMESTAMP` is needed.

**Describe the solution you'd like**
I know that parquet logical type `TIMESTAMP` can represent from about -290k year to 294k year with microsecond precision. Still it looks weird that timestamp is exported as string because of such rare case. Parquet supports `NULL` value, so i'd like to suggest that CockroachDB Type `TIMESTAMP`/`TIMESTAMPTZ` is exported as parquet logical type `TIMESTAMP` and null if not representable.
Not only `TIMESTAMP` but other date/time related types like `DATE`, `TIME`/`TIMETZ` could be handled as the same way since parquet logical type has corresponding types.

Jira issue: CRDB-23076

Epic CRDB-2260

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.