cockroachdb / cockroachdb/cockroach
sql/importer: unsigned parquet ints (UINT_32/UINT_64) silently reinterpreted as signed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
IMPORT never inspects the signedness of a parquet integer column's `IntLogicalType` — neither in validation nor in conversion ([read_import_parquet_logical.go](https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/importer/read_import_parquet_logical.go)). Conversion falls through to `tree.NewDInt(tree.DInt(v))`, which sign-extends the raw physical value. Unsigned values above the signed max are silently imported as negative numbers, and the job reports `succeeded`:
- UINT_32 `3000000000` → `-1294967296`
- UINT_64 `18446744073709551615` (max) → `-1`
The same corruption applies on the LIST→ARRAY and LIST→JSONB paths.
pyarrow writes pandas/arrow `uint32`/`uint64` columns with these logical types by default, so this is mainstream-writer silent corruption, not an exotic file shape.
**To Reproduce**
```python
import pyarrow as pa, pyarrow.parquet as pq
t = pa.table({
"u32": pa.array([3000000000], pa.uint32()),
"u64": pa.array([18446744073709551615], pa.uint64()),
})
pq.write_table(t, "uints.parquet")
```
```sql
CREATE TABLE t (u32 INT8, u64 INT8);
IMPORT INTO t PARQUET DATA ('nodelocal://1/uints.parquet'); -- succeeded
SELECT * FROM t;
-- u32 | u64
-- --------------+------
-- -1294967296 | -1
```
**Expected behavior**
- UINT_32 values fit in INT8 after widening through uint32 — they should import correctly.
- UINT_64 values above MaxInt64 don't fit in any CRDB integer type; the row should fail with an out-of-range error (or be routed to a DECIMAL target), never silently wrap to a negative value.
For comparison, CRDB's own CSV import of the same data errors cleanly (verified side-by-side):
```
error parsing row 1: parse "u64" as INT8: could not parse "18446744073709551615" as type int:
strconv.ParseInt: ... value out of range
```
and Postgres likewise rejects out-of-range integer input (`ERROR: bigint out of range`, SQLSTATE 22003) rather than wrapping. Parquet IMPORT is the outlier.
**Environment:**
- Reproduced on current master (26.3 dev), pyarrow-generated files. The affected code shipped in v26.2.0, so 26.2 is likely affected as well (not separately verified).
**Additional context**
Silent data corruption with a `succeeded` job; nothing flags the bad rows.
Jira issue: CRDB-65559
Epic CRDB-66113
Contributor guide
Research direction
Start in pkg/sql/importer/read_import_parquet_logical.go and run the provided pyarrow reproduction against Parquet IMPORT. Trace validation and conversion for UINT_32 and UINT_64, including the LIST-to-ARRAY and LIST-to-JSONB paths; done means values are preserved when representable and out-of-range UINT_64 values fail instead of wrapping, with coverage for these cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100