[server][lake] TableInfo exposes inherited Iceberg warehouse as a table property
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Fluss version
main (development)
### Please describe the bug 🐞
### Description
For an Iceberg-enabled Fluss table, `Admin#getTableInfo` (and consumers such as catalogs or UIs) exposes cluster-level Iceberg catalog options as table properties by adding a `table.` prefix.
In the reproduced setup, the Fluss cluster is configured with:
```yaml
datalake.format: iceberg
datalake.iceberg.type: glue
datalake.iceberg.warehouse: s3:///fluss/iceberg
```
an Iceberg-enabled table is reported with:
```
table.datalake.iceberg.warehouse=s3:///fluss/iceberg
```
However, an existing Iceberg table may be stored at an explicit location such as `s3:///fluss/native_implicit_day_pk_demo_v2`. The reported warehouse still points to `s3:///fluss/iceberg`, even though the table is stored somewhere else. Because Fluss exposes this value as a table property, users can easily mistake it for the table's actual location.
### How to reproduce
1. Use a Fluss cluster configured with `datalake.iceberg.warehouse: s3:///fluss/iceberg`.
2. Create `native_implicit_day_pk_demo_v2` in the same Glue catalog with the explicit table location `s3:///fluss/native_implicit_day_pk_demo_v2`.
3. Create or bind the matching Fluss table with `table.datalake.enabled=true`.
4. Call `Admin#getTableInfo` or inspect the table through a catalog/UI.
5. Observe that `table.datalake.iceberg.warehouse` contains the cluster-level default warehouse even though the Iceberg table uses a different location.
### Expected behavior
`TableInfo.properties` should not make inherited cluster-level catalog configuration look like persisted table properties. Inherited/resolved catalog options should be distinguishable from stored table properties, and a warehouse value should not be presented as the actual location of an existing Iceberg table.
### Current behavior and impact
`LakeStorageUtils#generateDefaultTableLakeOptions` prefixes cluster options with `table.`, and `TableRegistration#toTableInfo` overlays them when returning `TableInfo`.
This is misleading for users and metadata consumers. It also means the reported table properties can change when cluster configuration changes, without any `ALTER TABLE`, and the problem is visible when binding existing Iceberg tables with custom locations.
### Related history
- #411 introduced the requirement to expose lake storage information through table properties.
- #503 implemented the dynamic overlay and removed `describeLakeStorage`.
- #3601 added support for tolerating existing Iceberg lake tables, making custom-location tables a normal supported case.
### Solution
Separate persisted table properties from inherited/resolved lake catalog options.
Possible approaches:
- expose resolved lake catalog options through a dedicated metadata structure instead of merging them into `TableInfo.properties`; or
- retain the resolved options but mark their source explicitly and keep persisted table properties separate.
Add a regression test with a cluster warehouse and an existing Iceberg table whose explicit location differs from that warehouse.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing Admin#getTableInfo through TableRegistration#toTableInfo and inspect how LakeStorageUtils#generateDefaultTableLakeOptions supplies the overlay. Reproduce the case with a cluster warehouse and an existing Iceberg table using a different explicit location, then add the requested regression test. Done means persisted table properties remain separate from inherited or resolved catalog options and the warehouse is not misleadingly presented as the table location.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100