cockroachdb / cockroachdb/cockroach

sql/importer: string parquet column silently imports corrupt values into TIMESTAMPTZ target

Open
#172,328 1 comment 0 reactions 0 assignees View on GitHub
A-import branch-release-26.2 branch-release-26.3 C-bug O-agent O-qa T-sql-foundations
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

When `IMPORT INTO ... PARQUET DATA` reads a parquet column whose physical/logical type is a string (BYTE_ARRAY/UTF8) into a `TIMESTAMPTZ` column, the import **succeeds** and writes garbage values with no error or warning. Every imported value decodes to a timestamp a few seconds after the Unix epoch (roughly `len(string)/2` seconds), regardless of the string's content. Even a string that isn't a timestamp at all (`'garbage-not-a-ts'`) imports "successfully".

The same string value imports **correctly** into a `TIMESTAMP` column. The cause is a missing case in `convertBytesBasedOnTargetType` ([read_import_parquet_types.go](https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/importer/read_import_parquet_types.go)): it handles `types.TimestampFamily` (parses the string) but not `types.TimestampTZFamily`, so a TIMESTAMPTZ target falls through to the `default:` branch and returns a `DString`. Meanwhile the up-front type validation ([read_import_parquet_logical.go](https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/importer/read_import_parquet_logical.go), byte-array allowlist) explicitly permits BYTE_ARRAY → TimestampTZFamily, so the import proceeds and the string datum is value-encoded into the TIMESTAMPTZ column, producing corrupt rows.

This breaks round-tripping CockroachDB's **own** `EXPORT INTO PARQUET` output, because EXPORT writes TIMESTAMPTZ columns as strings.

**To Reproduce**

Pure-SQL repro on a single node (v26.2.x or master):

```sql
CREATE TABLE src (i INT PRIMARY KEY, ts TIMESTAMPTZ);
INSERT INTO src VALUES (1, '2024-06-15 12:34:56+00');
EXPORT INTO PARQUET 'nodelocal://1/rt/' FROM TABLE src;

CREATE TABLE dst (i INT PRIMARY KEY, ts TIMESTAMPTZ);
IMPORT INTO dst PARQUET DATA ('nodelocal://1/rt/*.parquet');
-- import reports success, 1 row

SELECT * FROM dst;
-- i | ts
-- ----+---------------------------
-- 1 | 1970-01-01 00:00:11+00 <-- silently corrupted
```

Any external string-typed parquet column reproduces it too, including non-timestamp content:

```python
import pyarrow as pa, pyarrow.parquet as pq
pq.write_table(pa.table({"ts": pa.array(["garbage-not-a-ts"], pa.string())}), "bad.parquet")
```

```sql
CREATE TABLE dst2 (ts TIMESTAMPTZ);
IMPORT INTO dst2 PARQUET DATA ('nodelocal://1/bad.parquet'); -- succeeds!
SELECT * FROM dst2; -- 1970-01-01 00:00:07.999999+00
```

**Expected behavior**

Either the string is parsed as a timestamp (matching the `TIMESTAMP` target behavior, and making EXPORT→IMPORT round trips work), or the import fails with a type-mismatch error. Silently writing corrupt data is the worst outcome.

**Additional data / screenshots**

- The fix is likely a `TimestampTZFamily` case in `convertBytesBasedOnTargetType` mirroring the existing `TimestampFamily` case (using `tree.ParseDTimestampTZ`).
- Defense in depth: nothing on the import path verifies that the converted datum's type matches the target column before KV encoding, which is why the stray `DString` is encoded rather than rejected. A check there would convert this class of bug from silent corruption into an error (separate issue to follow).

**Environment:**

- Reproduced on v26.2.3 and current master. Shipped in v26.2.0 (parquet IMPORT GA) → backport candidate for release-26.2.

**Additional context**

Silent data corruption on a supported import path; `IMPORT` reports success and no tooling flags the bad rows. Affects anyone importing parquet files with string-encoded timestamps into TIMESTAMPTZ columns — including files produced by CockroachDB's own EXPORT.

Jira issue: CRDB-65555

Epic CRDB-66113

Contributor guide

Open the contributing guide

Research direction

Start with pkg/sql/importer/read_import_parquet_types.go and convertBytesBasedOnTargetType, then compare its timestamp handling with the BYTE_ARRAY allowlist in read_import_parquet_logical.go. Run the SQL reproduction using EXPORT and IMPORT, plus the non-timestamp string case; done means TIMESTAMPTZ strings are parsed or rejected instead of silently producing corrupt rows.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.