[Bug] information_schema.tables on an external Iceberg catalog loads every table from the remote catalog (unconditional COMMENT fill)
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 520
Description
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Version
master (reproduced on a 2026-07-22 master build)
### What's Wrong?
Querying `information_schema.tables` while switched to an external Iceberg catalog (REST catalog in our case) issues one blocking `loadTable` against the remote catalog **for every table in the catalog**, even when the query only projects cheap columns like `TABLE_SCHEMA`/`TABLE_NAME`.
On a REST catalog with a few thousand tables, a single listing takes minutes and floods the catalog service with per-table GETs. Since the loads happen inside the `listTableStatus` thrift handler, clients with statement timeouts (BI tools, dashboards doing schema discovery) just see the query time out. Repeated runs pay the full cost again once the catalog metadata refresh interval passes.
Measured on our cluster: a catalog with 3,882 tables took 2m52s; one with 4,365 tables took 5m38s, with thousands of catalog GETs per scan.
jstack of the master FE during a `SELECT table_schema, table_name FROM information_schema.tables` scan:
```
at org.apache.iceberg.rest.RESTSessionCatalog.loadTable(RESTSessionCatalog.java:399)
...
at org.apache.doris.datasource.iceberg.IcebergUtils.getIcebergTable(IcebergUtils.java:943)
at org.apache.doris.datasource.iceberg.IcebergExternalTable.getIcebergTable(IcebergExternalTable.java:150)
at org.apache.doris.datasource.iceberg.IcebergExternalTable.properties(IcebergExternalTable.java:429)
at org.apache.doris.datasource.iceberg.IcebergExternalTable.getComment(IcebergExternalTable.java:155)
at org.apache.doris.service.FrontendServiceImpl.listTableStatus(FrontendServiceImpl.java:756)
```
Root cause is the interaction of two changes:
- #64263 made `IcebergExternalTable.getComment()` read the comment property from the table metadata, which requires loading the table from the remote catalog.
- #64933 introduced required-columns pruning in `listTableStatus`, guarding every status column (ENGINE, CREATE_TIME, TABLE_ROWS, ...) — but the `status.setComment(table.getComment())` fill remained unconditional.
So the one column whose fill is a remote round-trip is also the one column that is always computed.
### What You Expected?
A projection that does not include `TABLE_COMMENT` should not load table metadata from the remote catalog, the same way it already skips row counts and the other pruned status columns.
### How to Reproduce?
1. Create an Iceberg REST catalog with a large number of tables (a few hundred is enough to see it clearly).
2. `SWITCH` to the catalog and run `SELECT table_schema, table_name FROM information_schema.tables;`
3. Observe one `loadTable` request per table on the catalog service, and minutes of latency on large catalogs.
### Anything Else?
Fix appears to be a one-hunk change: guard the comment fill with the same `needTableStatusColumn(requiredColumns, "TABLE_COMMENT")` check used by the other columns. I will attach a PR.
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Research direction
Start in FrontendServiceImpl.java at listTableStatus, then read the existing requiredColumns checks around the status-column fills and the TABLE_COMMENT path. Reproduce with SELECT table_schema, table_name FROM information_schema.tables against an external Iceberg REST catalog, and confirm that the projection no longer triggers per-table metadata loads while TABLE_COMMENT queries still return comments.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100