apache / apache/fluss

After enabling the layered service, there is an error message when querying data

Open
#2,088 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Fluss version

0.8.0 (latest release)

### Please describe the bug 🐞

CREATE TABLE fluss_orders(
> order_id string,
> user_id int,
> sku_id string,
> amount decimal(10,2),
> order_time TIMESTAMP(0),
> `ptime` AS PROCTIME(),
> PRIMARY KEY (order_id) NOT ENFORCED
> ) WITH (
> 'table.datalake.enabled' = 'true',
> 'table.datalake.freshness' = '1m',
> 'paimon.file.format' = 'parquet',
> 'paimon.deletion-vectors.enabled' = 'true'
> );
===================================================
/opt/flink-1.20.3/bin/flink run \
-Dpipeline.name="My Fluss Tiering Service1" \
-Dexecution.checkpointing.interval=2m \
-Dexecution.checkpointing.timeout=3m \
-Dexecution.checkpointing.min-pause=1m \
-Dexecution.checkpointing.mode=EXACTLY_ONCE \
-Dparallelism.default=2 \
/opt/fluss-0.8.0-incubating/fluss-flink-tiering-0.8.0-incubating.jar \
--fluss.bootstrap.servers 192.168.40.102:9123 \
--datalake.format paimon \
--datalake.paimon.metastore hive \
--datalake.paimon.warehouse hdfs:///user/hive/warehouse \
--datalake.paimon.uri thrift://192.168.40.102:9083

2025-12-03 20:10:32
org.apache.flink.util.FlinkException: Global failure triggered by OperatorCoordinator for 'Source: fluss_orders[4] -> Calc[5] -> ConstraintEnforcer[6]' (operator cbc357ccb763df2852fee8c4fc7d55f2).
at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder$LazyInitializedCoordinatorContext.failJob(OperatorCoordinatorHolder.java:651)
at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$QuiesceableContext.failJob(RecreateOnResetOperatorCoordinator.java:259)
at org.apache.flink.runtime.source.coordinator.SourceCoordinatorContext.failJob(SourceCoordinatorContext.java:432)
at org.apache.flink.runtime.source.coordinator.SourceCoordinatorContext.handleUncaughtExceptionFromAsyncCall(SourceCoordinatorContext.java:445)
at org.apache.flink.util.ThrowableCatchingRunnable.run(ThrowableCatchingRunnable.java:42)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:842)
Caused by: org.apache.flink.util.FlinkRuntimeException: Failed to list splits for luxl_test.fluss_orders to read due to
at org.apache.fluss.flink.source.enumerator.FlinkSourceEnumerator.handleSplitsAdd(FlinkSourceEnumerator.java:679)
at org.apache.flink.runtime.source.coordinator.ExecutorNotifier.lambda$null$1(ExecutorNotifier.java:83)
at org.apache.flink.util.ThrowableCatchingRunnable.run(ThrowableCatchingRunnable.java:40)
... 6 more
Caused by: org.apache.flink.util.FlinkRuntimeException: Failed to generate hybrid lake fluss splits
at org.apache.fluss.flink.source.enumerator.FlinkSourceEnumerator.generateHybridLakeFlussSplits(FlinkSourceEnumerator.java:634)
at org.apache.fluss.flink.source.enumerator.FlinkSourceEnumerator.lambda$startInStreamModeForNonPartitionedTable$1(FlinkSourceEnumerator.java:323)
at org.apache.flink.runtime.source.coordinator.ExecutorNotifier.lambda$notifyReadyAsync$2(ExecutorNotifier.java:80)
... 6 more
Caused by: java.util.NoSuchElementException
at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1599)
at java.base/java.util.HashMap$ValueIterator.next(HashMap.java:1625)
at org.apache.fluss.flink.lake.LakeSplitGenerator.generateHybridLakeFlussSplits(LakeSplitGenerator.java:122)
at org.apache.fluss.flink.source.enumerator.FlinkSourceEnumerator.generateHybridLakeFlussSplits(FlinkSourceEnumerator.java:631)
... 8 more

=============================================================

[root@localhost ~]# hadoop fs -ls /user/hive/warehouse/luxl_test.db/fluss_orders
Found 4 items
drwxr-xr-x - root supergroup 0 2025-12-03 20:08 /user/hive/warehouse/luxl_test.db/fluss_orders/bucket-0
drwxr-xr-x - root supergroup 0 2025-12-03 20:08 /user/hive/warehouse/luxl_test.db/fluss_orders/manifest
drwxr-xr-x - root supergroup 0 2025-12-03 20:06 /user/hive/warehouse/luxl_test.db/fluss_orders/schema
drwxr-xr-x - root supergroup 0 2025-12-03 20:08 /user/hive/warehouse/luxl_test.db/fluss_orders/snapshot
[root@localhost ~]#

================================================================
conf/server.yaml
datalake.format: paimon
datalake.paimon.metastore: hive
datalake.paimon.uri: thrift://192.168.40.102:9083
datalake.paimon.warehouse: hdfs:///user/hive/warehouse

### Solution

_No response_

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the query using the provided CREATE TABLE, tiering-service command, and conf/server.yaml settings. Start at FlinkSourceEnumerator.java:634 and LakeSplitGenerator.java:122, then verify the hybrid lake split generation no longer throws NoSuchElementException and the table can be queried successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.