cockroachdb / cockroachdb/cockroach

sql/importer: ENUM-annotated parquet column imported into STRING target panics the node

Open
#172,331 0 comments 0 reactions 0 assignees View on GitHub
A-import branch-release-26.2 branch-release-26.3 C-bug O-agent O-qa T-sql-foundations
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

IMPORT's parquet validation admits ENUM-annotated BYTE_ARRAY columns into STRING (and other) targets via the generic byte-array allowlist in [read_import_parquet_logical.go](https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/importer/read_import_parquet_logical.go), but the converter unconditionally calls `tree.MakeDEnumFromLogicalRepresentation(targetType, ...)`. When the target type is not an ENUM (e.g. `STRING`), that reaches `types.T.EnumGetIdxOfLogical` → `EnumMetadata()`, which hits:

```
panic: use of enum metadata before hydration as an enum: string
```

The panic propagates out of the import worker goroutine through `ctxgroup.Wait` in `distImport` ([import_processor_planning.go](https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/importer/import_processor_planning.go)) with no recovery and **kills the node** (same crash mode as #170184).

ENUM→STRING is the natural mapping whenever the target table doesn't use a CockroachDB enum type. ENUM-annotated columns are written by parquet-mr for Avro/Thrift/Protobuf enum fields, so such files exist in the wild. (Importing an ENUM-annotated column into an actual ENUM target works fine.)

**To Reproduce**

pyarrow cannot write the parquet ENUM logical type, but the Go parquet library can. Self-contained generator (`go mod init repro && go get github.com/apache/arrow/go/v11 && go run .`):

```go
package main

import (
"os"

"github.com/apache/arrow/go/v11/parquet"
"github.com/apache/arrow/go/v11/parquet/file"
"github.com/apache/arrow/go/v11/parquet/schema"
)

func main() {
idNode, _ := schema.NewPrimitiveNode("id", parquet.Repetitions.Optional,
parquet.Types.Int64, -1, -1)
enumNode, _ := schema.NewPrimitiveNodeLogical("e", parquet.Repetitions.Optional,
schema.EnumLogicalType{}, parquet.Types.ByteArray, -1, -1)
root, _ := schema.NewGroupNode("schema", parquet.Repetitions.Required,
schema.FieldList{idNode, enumNode}, -1)

f, _ := os.Create("enum.parquet")
w := file.NewParquetWriter(f, root)
rgw := w.AppendRowGroup()

cw, _ := rgw.NextColumn()
cw.(*file.Int64ColumnChunkWriter).WriteBatch([]int64{1, 2}, []int16{1, 1}, nil)
cw.Close()

cw, _ = rgw.NextColumn()
cw.(*file.ByteArrayColumnChunkWriter).WriteBatch(
[]parquet.ByteArray{parquet.ByteArray("red"), parquet.ByteArray("blue")},
[]int16{1, 1}, nil)
cw.Close()
rgw.Close()
w.Close()
}
```

```sql
CREATE TABLE t (id INT PRIMARY KEY, e STRING);
IMPORT INTO t PARQUET DATA ('nodelocal://1/enum.parquet');
-- node dies:
-- panic: use of enum metadata before hydration as an enum: string
```

**Expected behavior**

When the target family isn't Enum, the ENUM payload should be treated as a plain string — the LIST→JSONB path already does exactly this for enum elements. At minimum, validation should reject the combination with a clean error. A node crash on user-supplied input is never acceptable.

**Environment:**

- Reproduced on current master (26.3 dev). The converter/validation code shipped in v26.2.0, so 26.2 is likely affected as well (not separately verified).

**Additional context**

Any user with IMPORT privileges can take down nodes with a small crafted (but spec-legal) parquet file.

Jira issue: CRDB-65558

Epic CRDB-66113

Contributor guide

Open the contributing guide

Research direction

Start in pkg/sql/importer/read_import_parquet_logical.go, then trace the converter into tree.MakeDEnumFromLogicalRepresentation and the worker path through distImport in import_processor_planning.go. Run the provided Go parquet generator and IMPORT reproduction. Done means an ENUM-annotated column targeting STRING is handled as a string or rejected cleanly, without a node panic.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.