Reserved Iceberg metadata table names in Polaris (`history`, `entries`, etc.)
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 522
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 137
Description
### Describe the bug
We have seen issues with Polaris and tables named after Iceberg metadata table names.
For example, when we try to create tables named as: `entries`, `history`, etc.
It looks like this is an Iceberg problem and not directly a Polaris one.
**Upon Creation**
Polaris will call `Catalog::tableExists(...)`, which internally calls `BaseMetastoreCatalog::loadTable(...)`. If that call returns an exception then the table is considered non-existent.
The problem, however, is that `loadTable` will try to resolve the table as a metadata table if it's not found and if the table name matches one of the reserved metadata keywords defined [here](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/MetadataTableType.java), it will try to call `loadMetadataTable` using the namespace as a table name, and this will fail.
This issue is also open on the iceberg community:
https://github.com/apache/iceberg/issues/10550
**Possible Fix**
I understand that we want to keep `tableExists` implementation-agnostic and use `loadTable` in a `try/catch`, but the problem is that loadTable will try to resolve it as a metadata table if it matches one of these words. Since we do, however, have the `IcebergCatalog` class, we could simply override tableExists with the following:
```
public boolean tableExists(TableIdentifier identifier) {
if (isValidIdentifier(identifier)) {
return newTableOps(identifier).current() != null;
}
return false;
}
```
This would allow us to bypass the problem, since we only care about actual tables when we call this method and not metadata tables.
### To Reproduce
Build a table named `history`
### Actual Behavior
Fails with unclear error.
### Expected Behavior
Should accept it.
### Additional context
Clashes with metadata table names.
### System information
_No response_
Contributor guide
Research direction
Read IcebergCatalog.tableExists and the BaseMetastoreCatalog.loadTable path first, using MetadataTableType for context. Reproduce the issue by building a table named history or entries, then verify that creation succeeds without metadata-table resolution failing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100