cockroachdb / cockroachdb/cockroach
sql/importer: string parquet column silently imports corrupt values into TIMESTAMPTZ target
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
When `IMPORT INTO ... PARQUET DATA` reads a parquet column whose physical/logical type is a string (BYTE_ARRAY/UTF8) into a `TIMESTAMPTZ` column, the import **succeeds** and writes garbage values with no error or warning. Every imported value decodes to a timestamp a few seconds after the Unix epoch (roughly `len(string)/2` seconds), regardless of the string's content. Even a string that isn't a timestamp at all (`'garbage-not-a-ts'`) imports "successfully".
The same string value imports **correctly** into a `TIMESTAMP` column. The cause is a missing case in `convertBytesBasedOnTargetType` ([read_import_parquet_types.go](https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/importer/read_import_parquet_types.go)): it handles `types.TimestampFamily` (parses the string) but not `types.TimestampTZFamily`, so a TIMESTAMPTZ target falls through to the `default:` branch and returns a `DString`. Meanwhile the up-front type validation ([read_import_parquet_logical.go](https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/importer/read_import_parquet_logical.go), byte-array allowlist) explicitly permits BYTE_ARRAY → TimestampTZFamily, so the import proceeds and the string datum is value-encoded into the TIMESTAMPTZ column, producing corrupt rows.
This breaks round-tripping CockroachDB's **own** `EXPORT INTO PARQUET` output, because EXPORT writes TIMESTAMPTZ columns as strings.
**To Reproduce**
Pure-SQL repro on a single node (v26.2.x or master):
```sql
CREATE TABLE src (i INT PRIMARY KEY, ts TIMESTAMPTZ);
INSERT INTO src VALUES (1, '2024-06-15 12:34:56+00');
EXPORT INTO PARQUET 'nodelocal://1/rt/' FROM TABLE src;
CREATE TABLE dst (i INT PRIMARY KEY, ts TIMESTAMPTZ);
IMPORT INTO dst PARQUET DATA ('nodelocal://1/rt/*.parquet');
-- import reports success, 1 row
SELECT * FROM dst;
-- i | ts
-- ----+---------------------------
-- 1 | 1970-01-01 00:00:11+00 <-- silently corrupted
```
Any external string-typed parquet column reproduces it too, including non-timestamp content:
```python
import pyarrow as pa, pyarrow.parquet as pq
pq.write_table(pa.table({"ts": pa.array(["garbage-not-a-ts"], pa.string())}), "bad.parquet")
```
```sql
CREATE TABLE dst2 (ts TIMESTAMPTZ);
IMPORT INTO dst2 PARQUET DATA ('nodelocal://1/bad.parquet'); -- succeeds!
SELECT * FROM dst2; -- 1970-01-01 00:00:07.999999+00
```
**Expected behavior**
Either the string is parsed as a timestamp (matching the `TIMESTAMP` target behavior, and making EXPORT→IMPORT round trips work), or the import fails with a type-mismatch error. Silently writing corrupt data is the worst outcome.
**Additional data / screenshots**
- The fix is likely a `TimestampTZFamily` case in `convertBytesBasedOnTargetType` mirroring the existing `TimestampFamily` case (using `tree.ParseDTimestampTZ`).
- Defense in depth: nothing on the import path verifies that the converted datum's type matches the target column before KV encoding, which is why the stray `DString` is encoded rather than rejected. A check there would convert this class of bug from silent corruption into an error (separate issue to follow).
**Environment:**
- Reproduced on v26.2.3 and current master. Shipped in v26.2.0 (parquet IMPORT GA) → backport candidate for release-26.2.
**Additional context**
Silent data corruption on a supported import path; `IMPORT` reports success and no tooling flags the bad rows. Affects anyone importing parquet files with string-encoded timestamps into TIMESTAMPTZ columns — including files produced by CockroachDB's own EXPORT.
Jira issue: CRDB-65555
Epic CRDB-66113
Contributor guide
Research direction
Start with pkg/sql/importer/read_import_parquet_types.go and convertBytesBasedOnTargetType, then compare its timestamp handling with the BYTE_ARRAY allowlist in read_import_parquet_logical.go. Run the SQL reproduction using EXPORT and IMPORT, plus the non-timestamp string case; done means TIMESTAMPTZ strings are parsed or rejected instead of silently producing corrupt rows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100