cockroachdb / cockroachdb/cockroach

sql/importer: parquet LIST<timestamp> cannot be imported into TIMESTAMP[]

Open
#172,336 0 comments 0 reactions 0 assignees View on GitHub
A-import branch-release-26.3 C-bug O-agent O-qa T-sql-foundations
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

Importing a parquet LIST of timestamps into a `TIMESTAMP[]` column fails:

```
error parsing row 1: appending LIST element 0: cannot append TIMESTAMPTZ to array containing TIMESTAMP
```

Root cause: all four timestamp converters (`convertTimestampMillis/Micros/NanosFromInt64`, `convertTimestampFromInt96` in [read_import_parquet_types.go](https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/importer/read_import_parquet_types.go)) unconditionally return `DTimestampTZ` regardless of the target type. The flat-column path happens to tolerate the datum-kind mismatch (TIMESTAMP and TIMESTAMPTZ share a value-encoding wire format), but the LIST path appends through `tree.DArray.Append`, which type-checks strictly and errors. INT96 list elements hit the same wall.

Importing into `TIMESTAMPTZ[]` works (workaround).

**To Reproduce**

```python
import pyarrow as pa, pyarrow.parquet as pq, pandas as pd
ts = [pd.Timestamp("2024-01-01"), pd.Timestamp("2024-06-15 12:34:56")]
t = pa.table({"a": [1], "tss": pa.array([ts], pa.list_(pa.timestamp("us")))})
pq.write_table(t, "listts.parquet")
```

```sql
CREATE TABLE t (a INT PRIMARY KEY, tss TIMESTAMP[]);
IMPORT INTO t PARQUET DATA ('nodelocal://1/listts.parquet');
-- ERROR: ... cannot append TIMESTAMPTZ to array containing TIMESTAMP
```

**Expected behavior**

The converters should produce a `DTimestamp` when the target element type is TIMESTAMP (this would also remove the flat path's reliance on the accidental wire-format overlap).

**Environment:**

- Reproduced on current master. LIST import is new in 26.3.

**Additional context**

Found during 26.3 QA of the new LIST support; the clean error means no corruption, just an unusable type combination.

Jira issue: CRDB-65563

Epic CRDB-66113

Contributor guide

Open the contributing guide

Research direction

Start in pkg/sql/importer/read_import_parquet_types.go and inspect the four timestamp converter functions named in the issue, along with the LIST path that calls tree.DArray.Append. Reproduce the provided parquet and SQL example, then verify that importing into TIMESTAMP[] succeeds while the existing TIMESTAMPTZ[] behavior remains valid.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.