cockroachdb / cockroachdb/cockroach
sql/importer: no way to set crdb_region via parquet IMPORT on REGIONAL BY ROW tables (CSV can)
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
On a REGIONAL BY ROW table there is no way to import explicit per-row regions from a parquet file:
- Explicit targeting is rejected: `IMPORT INTO rbr (crdb_region, a, b) PARQUET DATA (...)` fails with `target column "crdb_region" not found in table schema` — while the identical CSV import succeeds and places each row in its named region. Parquet's target-column resolution apparently skips hidden columns.
- With auto-mapping, a `crdb_region` column present in the parquet file is silently ignored and the gateway-region default is applied.
Net: CSV can do explicit per-row region placement, parquet cannot — a format parity gap that matters when migrating multi-region data.
(The default path itself works: importing a file *without* `crdb_region` correctly lands all rows in the gateway region.)
**To Reproduce**
On a multi-region cluster (`cockroach demo --nodes=3 --demo-locality=region=us-east1:region=us-west1:region=europe-west1`):
```sql
CREATE DATABASE mr PRIMARY REGION "us-east1" REGIONS "us-west1", "europe-west1";
USE mr;
CREATE TABLE rbr (a INT PRIMARY KEY, b STRING) LOCALITY REGIONAL BY ROW;
```
```python
import pyarrow as pa, pyarrow.parquet as pq
t = pa.table({"crdb_region": ["us-west1", "europe-west1"], "a": [1, 2], "b": ["x", "y"]})
pq.write_table(t, "rbr.parquet")
```
```sql
IMPORT INTO rbr (crdb_region, a, b) PARQUET DATA ('nodelocal://1/rbr.parquet');
-- ERROR: target column "crdb_region" not found in table schema
IMPORT INTO rbr PARQUET DATA ('nodelocal://1/rbr.parquet');
-- succeeds, but crdb_region column ignored; all rows in gateway region
```
The same explicit-target CSV import succeeds and honors the per-row regions.
**Expected behavior**
Parity with CSV: explicit targeting of `crdb_region` resolves the hidden column and honors file values. Bonus: auto-mapping should either honor a `crdb_region` file column or (in strict mode) call out that it's being ignored, rather than silently dropping it.
**Environment:**
- Current master (26.3 dev), 3-region demo cluster.
Jira issue: CRDB-65569
Epic CRDB-66113
Contributor guide
Research direction
Start by reproducing the documented multi-region demo and the explicit-target and auto-mapped PARQUET imports. Read the SQL IMPORT PARQUET target-column resolution alongside the working CSV path; done means explicit crdb_region targeting honors per-row regions, with coverage for the described behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100