apache / apache/fluss

[Server] Missing table_bucket_lakeTiering Metrics for Lake-Enabled Tables

Open
#2,422 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Fluss version

0.8.0 (latest release)

### Please describe the bug 🐞

When enabling lake functionality after table creation via ALTER TABLE, two issues occur:
1. The table_bucket_lakeTiering monitoring metrics are not exposed, even though the table is successfully enabled for lake tiering.
2. Attempting to set or update the `table.datalake.freshness` option via ALTER TABLE fails with an unsupported operation error.

Steps to Reproduce
```sql
CREATE TABLE `fluss-test`.fluss.fluss_order_tiering_test_metrics1 (...);

ALTER TABLE `fluss-test`.fluss.fluss_order_tiering_test_metrics1
SET ('table.datalake.enabled' = 'true');

-- Observation: The table becomes lake-enabled, but no table_bucket_lakeTiering metrics appear in monitoring.

ALTER TABLE `fluss-test`.fluss.fluss_order_tiering_test_metrics1
SET (
'table.datalake.enabled' = 'true',
'table.datalake.freshness' = '30s'
);

-- Error:Caused by: org.apache.fluss.exception.InvalidAlterTableException: The option 'table.datalake.freshness' is not supported to alter yet.
```

```
Execution statement throws exception, rootCauseMessage = FlinkSQLException: org.apache.flink.table.api.TableException: Could not execute AlterTable in path `fluss-test`.`fluss`.`fluss_order_tiering_test_metrics1`. Failed to alter table fluss.fluss_order_tiering_test_metrics1 in fluss-test
at org.apache.flink.table.catalog.CatalogManager.execute(CatalogManager.java:1952)
at org.apache.flink.table.catalog.CatalogManager.alterTable(CatalogManager.java:1454)
at org.apache.flink.table.operations.ddl.AlterTableChangeOperation.execute(AlterTableChangeOperation.java:160)
at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:1218)
at org.apache.flink.table.gateway.service.operation.OperationExecutor.callOperation(OperationExecutor.java:797)
at org.apache.flink.table.gateway.service.operation.OperationExecutor.executeOperation(OperationExecutor.java:537)
at org.apache.flink.table.gateway.service.operation.OperationExecutor.lambda$executeStatement$1(OperationExecutor.java:277)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
at org.apache.flink.table.gateway.service.context.SqlGatewaySecurityContext.runSecured(SqlGatewaySecurityContext.java:73)
at org.apache.flink.table.gateway.service.operation.OperationExecutor.executeStatement(OperationExecutor.java:231)
at org.apache.flink.table.gateway.service.operation.OperationExecutor.executeStatement(OperationExecutor.java:217)
at org.apache.flink.table.gateway.service.SqlGatewayServiceImpl.lambda$executeStatement$1(SqlGatewayServiceImpl.java:283)
at org.apache.flink.table.gateway.service.operation.OperationManager.lambda$submitOperation$1(OperationManager.java:137)
at org.apache.flink.table.gateway.service.operation.OperationManager$Operation.lambda$run$0(OperationManager.java:288)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:984)
Caused by: org.apache.flink.table.catalog.exceptions.CatalogException: Failed to alter table fluss.fluss_order_tiering_test_metrics1 in fluss-test
at org.apache.fluss.flink.catalog.FlinkCatalog.alterTable(FlinkCatalog.java:506)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.flink.table.gateway.vvr.module.delegation.CatalogFactoryWrapper$BuiltInFactoryInvocationHandler.invoke0(CatalogFactoryWrapper.java:185)
at org.apache.flink.table.gateway.vvr.module.delegation.FactoryWrapperBase$ArtifactInvocationHandler.invoke(FactoryWrapperBase.java:106)
at com.sun.proxy.$Proxy136.alterTable(Unknown Source)
at org.apache.flink.table.catalog.CatalogManager.lambda$alterTable$30(CatalogManager.java:1457)
at org.apache.flink.table.catalog.CatalogManager.execute(CatalogManager.java:1944)
... 22 more
Caused by: org.apache.fluss.exception.InvalidAlterTableException: The option 'table.datalake.freshness' is not supported to alter yet.
```

### Solution

The `table.datalake.freshness` configuration should be alterable after table creation

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the ALTER TABLE statements and start in org.apache.fluss.flink.catalog.FlinkCatalog.alterTable, where the reported InvalidAlterTableException is surfaced. Trace how table.datalake.freshness changes are validated and how table_bucket_lakeTiering metrics are exposed after lake enablement. Done means both ALTER TABLE operations work and the expected metrics appear for a lake-enabled table.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
backend, databases, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.