cockroachdb / cockroachdb/cockroach
sql/importer: parquet LIST<timestamp> cannot be imported into TIMESTAMP[]
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
Importing a parquet LIST of timestamps into a `TIMESTAMP[]` column fails:
```
error parsing row 1: appending LIST element 0: cannot append TIMESTAMPTZ to array containing TIMESTAMP
```
Root cause: all four timestamp converters (`convertTimestampMillis/Micros/NanosFromInt64`, `convertTimestampFromInt96` in [read_import_parquet_types.go](https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/importer/read_import_parquet_types.go)) unconditionally return `DTimestampTZ` regardless of the target type. The flat-column path happens to tolerate the datum-kind mismatch (TIMESTAMP and TIMESTAMPTZ share a value-encoding wire format), but the LIST path appends through `tree.DArray.Append`, which type-checks strictly and errors. INT96 list elements hit the same wall.
Importing into `TIMESTAMPTZ[]` works (workaround).
**To Reproduce**
```python
import pyarrow as pa, pyarrow.parquet as pq, pandas as pd
ts = [pd.Timestamp("2024-01-01"), pd.Timestamp("2024-06-15 12:34:56")]
t = pa.table({"a": [1], "tss": pa.array([ts], pa.list_(pa.timestamp("us")))})
pq.write_table(t, "listts.parquet")
```
```sql
CREATE TABLE t (a INT PRIMARY KEY, tss TIMESTAMP[]);
IMPORT INTO t PARQUET DATA ('nodelocal://1/listts.parquet');
-- ERROR: ... cannot append TIMESTAMPTZ to array containing TIMESTAMP
```
**Expected behavior**
The converters should produce a `DTimestamp` when the target element type is TIMESTAMP (this would also remove the flat path's reliance on the accidental wire-format overlap).
**Environment:**
- Reproduced on current master. LIST import is new in 26.3.
**Additional context**
Found during 26.3 QA of the new LIST support; the clean error means no corruption, just an unusable type combination.
Jira issue: CRDB-65563
Epic CRDB-66113
Contributor guide
Research direction
Start in pkg/sql/importer/read_import_parquet_types.go and inspect the four timestamp converter functions named in the issue, along with the LIST path that calls tree.DArray.Append. Reproduce the provided parquet and SQL example, then verify that importing into TIMESTAMP[] succeeds while the existing TIMESTAMPTZ[] behavior remains valid.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 75/100