cockroachdb / cockroachdb/cockroach

sql/importer: UUID-annotated parquet column imported into STRING/BYTES poisons the table; auto stats then crash-loops the node

Open
#172,330 0 comments 0 reactions 0 assignees View on GitHub
A-import branch-release-26.2 branch-release-26.3 C-bug O-agent O-qa T-sql-foundations
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

IMPORT's parquet type validation admits UUID-annotated FIXED_LEN_BYTE_ARRAY(16) columns into `STRING` or `BYTES` targets (FLBA allowlist in [read_import_parquet_logical.go](https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/importer/read_import_parquet_logical.go)), but the converter returns a `DUuid` datum regardless of the target type. A `DUuid`'s value encoding is not decodable as STRING/BYTES, so the import writes rows that can never be read. The consequences chain from bad to worse — all empirically verified:

1. The IMPORT job reports `succeeded`.
2. The post-import INSPECT job also `succeeded` (it only checks index consistency).
3. Every subsequent read of the table fails: `XX000 internal error: ... slice bounds out of range [17:15]` in `encoding.DecodeUntaggedBytesValue` (the vectorized engine catches the panic and turns it into an internal error).
4. About a minute later, AUTO CREATE STATS samples the table through the row-based `samplerProcessor`, which has **no panic recovery** (#112072) — the same slice-bounds panic takes down the **whole node**.
5. On a cluster with persistent stores this is a **crash loop**: auto stats re-fires after every restart for as long as the table exists.

INTERVAL-annotated FLBA is admitted into STRING/BYTES the same way and converts to `DInterval` — same failure class.

UUID→STRING is a natural mapping whenever the target table stores UUIDs as text, so this is easy to hit with ordinary pyarrow output.

**To Reproduce**

```python
import pyarrow as pa, pyarrow.parquet as pq, uuid
arr = pa.array([uuid.uuid4().bytes for _ in range(2)], pa.uuid())
pq.write_table(pa.table({"i": [1, 2], "u": arr}), "uuid.parquet")
```

```sql
CREATE TABLE t (i INT PRIMARY KEY, u STRING);
IMPORT INTO t PARQUET DATA ('nodelocal://1/uuid.parquet'); -- succeeded, 2 rows
SELECT * FROM t;
-- ERROR: internal error: ... slice bounds out of range [17:15]
-- ...then wait ~1 minute for auto stats:
```

```
panic: runtime error: slice bounds out of range [17:15]

goroutine ... ["job":"AUTO CREATE STATS id=..."]:
github.com/cockroachdb/cockroach/pkg/util/encoding.DecodeUntaggedBytesValue(...)
github.com/cockroachdb/cockroach/pkg/sql/rowenc/valueside.DecodeUntaggedDatum(...)
github.com/cockroachdb/cockroach/pkg/sql/rowenc.(*EncDatum).EnsureDecoded(...)
github.com/cockroachdb/cockroach/pkg/sql/stats.(*SampleReservoir).copyRow(...)
pkg/sql/stats/row_sampling.go:260
```

Same result with a `u BYTES` target. Dropping the table stops the crash loop.

**Expected behavior**

Either the converter honors the target type (encode the UUID as its string/byte representation for STRING/BYTES targets), or validation rejects the combination. Under no circumstances should IMPORT write rows that cannot be decoded.

**Additional data / screenshots**

- The underlying enabler is that nothing on the import path checks that the converted datum's kind matches the target column family before KV encoding (separate hardening issue to follow).
- The samplerProcessor's lack of panic recovery converts *any* undecodable row — from this bug or any other corruption source — into a node crash; that is tracked in #112072, and this bug is a fresh, easy repro for it (with the added crash-loop twist via auto stats re-firing after restart).

**Environment:**

- Reproduced on current master (26.3 dev). The affected converter/validation code shipped in v26.2.0, so 26.2 is likely affected as well (not separately verified).

**Additional context**

Import "succeeds", INSPECT passes, and the first symptom a user sees is unreadable data followed by a node crash loop driven by background auto stats. Recovery requires figuring out which table is poisoned and dropping it.

Jira issue: CRDB-65557

Epic CRDB-66113

Contributor guide

Open the contributing guide

Research direction

Start in pkg/sql/importer/read_import_parquet_logical.go and trace the parquet logical-type validation and conversion path for UUID- and INTERVAL-annotated FLBA columns targeting STRING or BYTES. Reproduce the issue with the provided pyarrow and IMPORT commands, then add coverage showing that the combination is rejected or safely converted without unreadable rows or an auto-stats crash loop.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.