apache / apache/doris

[Bug] CREATE ROW POLICY on a view over an external catalog fails "column not exist: <col>" after FE metadata replay

Open
#67,625 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

## Description
`CREATE ROW POLICY ... ON USING ( ...)` fails with
`column not exist: ` when `` is a view (in the `internal`
catalog) whose body reads a table in an **external** catalog (e.g. an Iceberg
REST catalog) — but only **after an FE metadata replay** (FE restart / failover).
A freshly created view of the same shape works. Meanwhile `DESCRIBE`, `SELECT
`, `information_schema.columns`, and `SHOW CREATE VIEW` all resolve ``
correctly on the very same view, and the row policy would enforce fine at query
time — so only the CREATE-time validation is wrong.

## Reproduce
1. Create an external catalog (Iceberg REST) `ext`, with a table `ext.db.t(col ...)`.
2. In the internal catalog, create a view over it:
`CREATE VIEW internal.d.v AS SELECT ext.db.t.col FROM ext.db.t;`
3. `CREATE ROW POLICY p ON internal.d.v AS RESTRICTIVE TO ROLE r USING (col IS NOT NULL);`
-> succeeds.
4. Restart the FE (or trigger a failover) so `internal.d.v` is rebuilt from the
edit log / image (metadata replay).
5. `CREATE ROW POLICY p2 ON internal.d.v AS RESTRICTIVE TO ROLE r USING (col IS NOT NULL);`
-> **fails: `column not exist: col`**, while `DESCRIBE internal.d.v` still lists `col`.

## Root cause
`CreatePolicyCommand#validate()` checks column existence with
`tableIf.getColumn(slot.getName())`. `Table#getColumn(name)` reads the
`nameToColumn` map, which the `Table` constructor keys by
`Column#getDefineName()`. For a view over an external catalog, metadata replay
reconstructs the view's columns with a `defineName` that differs from the plain
column `name` (the qualified source reference), so `getColumn(name)` misses a
column that is present in `getFullSchema()`. `DESCRIBE`, query planning, and the
row-policy runtime (`LogicalCheckPolicy` binds the stored predicate against the
analyzed plan output, not `getColumn`) all resolve the column, which is why only
the CREATE-time guard fails, and only after replay.

## Fix
Resolve the predicate columns against `getFullSchema()` (case-insensitive) — the
same source `DESCRIBE`/query planning use, and a superset of the `nameToColumn`
map, so the guard never rejects a column `getColumn` would have accepted. A PR
follows.

## Version
Reproduces on master (the `getColumn`-based check is present on the current tip).

Contributor guide

Open the contributing guide

Research direction

Start at CreatePolicyCommand#validate(), then compare Table#getColumn(name) with the replayed view schema from getFullSchema(). Read LogicalCheckPolicy to understand the runtime binding and inspect the metadata replay path for reconstructed view columns. Done means CREATE ROW POLICY resolves the predicate column after FE restart or failover, while the existing DESCRIBE and query behavior remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
databases, security
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.