ClickHouse / ClickHouse/ClickHouse
Iceberg table with a geometry column becomes unreadable after server restart; allow_experimental_geo_types_in_iceberg is latched at ATTACH, not checked per query
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
**Describe what's wrong**
The experimental gate `allow_experimental_geo_types_in_iceberg` is evaluated against the context captured when the Iceberg storage object is constructed (CREATE / ATTACH / server startup), not against the query context. Two user-visible consequences:
1. **An Iceberg table with a `geometry` column becomes permanently unreadable after a server restart** (or after any `DETACH`/`ATTACH` without the flag in the session): every subsequent `SELECT` throws `Code: 36` — *even when the query itself sets `allow_experimental_geo_types_in_iceberg = 1`*. The only cure is a manual `DETACH TABLE` + `ATTACH TABLE` with the flag enabled in the session.
2. Conversely, a table attached with the flag enabled is freely readable by queries that do **not** set the flag, so the gate is not actually enforced at read time.
The gate lives in `IcebergSchemaProcessor` (`src/Storages/ObjectStorage/DataLakes/Iceberg/SchemaProcessor.cpp`, the `allow_geo_parser` member checked when parsing a `geometry`/`geography` field), and `allow_geo_parser` is fixed from the settings of whichever context built the metadata object. Server startup attaches tables with server-default settings, so restart wedges the table. This is the same constructor-latched-guard shape as #115044, but with a worse consequence: a restart breaks working tables.
**How to reproduce**
ClickHouse 26.8.1.1516 (current master).
```sql
SET allow_experimental_geo_types_in_iceberg = 1;
CREATE TABLE geo (id Int64, g Geometry) ENGINE = IcebergLocal('/var/lib/clickhouse/user_files/geo/', 'Parquet');
INSERT INTO geo SELECT 1, readWKT('POINT(1 2)') SETTINGS allow_insert_into_iceberg = 1;
SELECT id, wkt(g) FROM geo; -- works: 1 POINT(1 2)
```
Now restart the server (or, in the same session: `SET allow_experimental_geo_types_in_iceberg = 0; DETACH TABLE geo; ATTACH TABLE geo;`), then:
```sql
SELECT id, wkt(g) FROM geo SETTINGS allow_experimental_geo_types_in_iceberg = 1;
```
```
Code: 36. DB::Exception: Using geometry/geography types is not allowed without enabled allow_experimental_geo_types_in_iceberg flag. (BAD_ARGUMENTS)
```
The flag on the query does not help. Recovery:
```sql
SET allow_experimental_geo_types_in_iceberg = 1;
DETACH TABLE geo;
ATTACH TABLE geo;
SELECT id, wkt(g) FROM geo; -- works again, and keeps working even for queries WITHOUT the flag
```
**Expected behavior**
Either the gate is enforced per query (a read without the flag fails, a read with the flag succeeds, regardless of how the storage was constructed), or — if the gate is intentionally a table-level property — it should be captured at CREATE and survive ATTACH/restart. A working table must not become unreadable after a restart while the documented enabling flag is powerless to read it.
**Additional context**
`Related: https://github.com/ClickHouse/ClickHouse/issues/115044` (the same constructor-only-guard shape on the Delta Lake CDF settings).
Found by an automatic optimizer-testing framework (differential testing of optimizer settings, query plans, and equivalent rewrites).
Contributor guide
Assessment
This issue has not been assessed yet.