cockroachdb / cockroachdb/cockroach

sql/importer: unsigned parquet ints (UINT_32/UINT_64) silently reinterpreted as signed

Open
#172,332 0 comments 0 reactions 0 assignees View on GitHub
A-import branch-release-26.2 branch-release-26.3 C-bug O-agent O-qa T-sql-foundations
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

IMPORT never inspects the signedness of a parquet integer column's `IntLogicalType` — neither in validation nor in conversion ([read_import_parquet_logical.go](https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/importer/read_import_parquet_logical.go)). Conversion falls through to `tree.NewDInt(tree.DInt(v))`, which sign-extends the raw physical value. Unsigned values above the signed max are silently imported as negative numbers, and the job reports `succeeded`:

- UINT_32 `3000000000` → `-1294967296`
- UINT_64 `18446744073709551615` (max) → `-1`

The same corruption applies on the LIST→ARRAY and LIST→JSONB paths.

pyarrow writes pandas/arrow `uint32`/`uint64` columns with these logical types by default, so this is mainstream-writer silent corruption, not an exotic file shape.

**To Reproduce**

```python
import pyarrow as pa, pyarrow.parquet as pq
t = pa.table({
"u32": pa.array([3000000000], pa.uint32()),
"u64": pa.array([18446744073709551615], pa.uint64()),
})
pq.write_table(t, "uints.parquet")
```

```sql
CREATE TABLE t (u32 INT8, u64 INT8);
IMPORT INTO t PARQUET DATA ('nodelocal://1/uints.parquet'); -- succeeded
SELECT * FROM t;
-- u32 | u64
-- --------------+------
-- -1294967296 | -1
```

**Expected behavior**

- UINT_32 values fit in INT8 after widening through uint32 — they should import correctly.
- UINT_64 values above MaxInt64 don't fit in any CRDB integer type; the row should fail with an out-of-range error (or be routed to a DECIMAL target), never silently wrap to a negative value.

For comparison, CRDB's own CSV import of the same data errors cleanly (verified side-by-side):

```
error parsing row 1: parse "u64" as INT8: could not parse "18446744073709551615" as type int:
strconv.ParseInt: ... value out of range
```

and Postgres likewise rejects out-of-range integer input (`ERROR: bigint out of range`, SQLSTATE 22003) rather than wrapping. Parquet IMPORT is the outlier.

**Environment:**

- Reproduced on current master (26.3 dev), pyarrow-generated files. The affected code shipped in v26.2.0, so 26.2 is likely affected as well (not separately verified).

**Additional context**

Silent data corruption with a `succeeded` job; nothing flags the bad rows.

Jira issue: CRDB-65559

Epic CRDB-66113

Contributor guide

Open the contributing guide

Research direction

Start in pkg/sql/importer/read_import_parquet_logical.go and run the provided pyarrow reproduction against Parquet IMPORT. Trace validation and conversion for UINT_32 and UINT_64, including the LIST-to-ARRAY and LIST-to-JSONB paths; done means values are preserved when representable and out-of-range UINT_64 values fail instead of wrapping, with coverage for these cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.