apache / apache/polaris

Reserved Iceberg metadata table names in Polaris (`history`, `entries`, etc.)

Open
#1,771 1 comment 1 reaction 0 assignees View on GitHub
bug
Dominant language
Java
Stars
2.1k
Forks
522
Avg merge
1d 22h
Merged PRs (30d)
137

Description

### Describe the bug

We have seen issues with Polaris and tables named after Iceberg metadata table names.
For example, when we try to create tables named as: `entries`, `history`, etc.

It looks like this is an Iceberg problem and not directly a Polaris one.

**Upon Creation**
Polaris will call `Catalog::tableExists(...)`, which internally calls `BaseMetastoreCatalog::loadTable(...)`. If that call returns an exception then the table is considered non-existent.

The problem, however, is that `loadTable` will try to resolve the table as a metadata table if it's not found and if the table name matches one of the reserved metadata keywords defined [here](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/MetadataTableType.java), it will try to call `loadMetadataTable` using the namespace as a table name, and this will fail.

This issue is also open on the iceberg community:
https://github.com/apache/iceberg/issues/10550

**Possible Fix**
I understand that we want to keep `tableExists` implementation-agnostic and use `loadTable` in a `try/catch`, but the problem is that loadTable will try to resolve it as a metadata table if it matches one of these words. Since we do, however, have the `IcebergCatalog` class, we could simply override tableExists with the following:

```
public boolean tableExists(TableIdentifier identifier) {
if (isValidIdentifier(identifier)) {
return newTableOps(identifier).current() != null;
}
return false;
}
```

This would allow us to bypass the problem, since we only care about actual tables when we call this method and not metadata tables.

### To Reproduce

Build a table named `history`

### Actual Behavior

Fails with unclear error.

### Expected Behavior

Should accept it.

### Additional context

Clashes with metadata table names.

### System information

_No response_

Contributor guide

Open the contributing guide

Research direction

Read IcebergCatalog.tableExists and the BaseMetastoreCatalog.loadTable path first, using MetadataTableType for context. Reproduce the issue by building a table named history or entries, then verify that creation succeeds without metadata-table resolution failing.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.