[Tiering] Trigger job failover when coordinator epoch changes
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Fluss version
0.9.0 (latest release)
### Please describe the bug 🐞
TieringSourceEnumerator records the coordinator epoch only at startup. After a coordinator failover or upgrade, it continues tiering with the stale epoch. The coordinator rejects subsequent heartbeats, causing tiering tasks to time out and the failuresTotal metric to keep increasing.
### Solution
Validate the coordinator epoch in every heartbeat response. If it differs from the startup epoch, throw an exception to trigger a Flink job failover. The restarted enumerator will register again with the latest coordinator epoch.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating TieringSourceEnumerator and the code that processes coordinator heartbeat responses. Verify how the startup epoch is stored and how heartbeat failures are handled. Done means a changed coordinator epoch triggers a Flink job failover, allowing the restarted enumerator to register with the latest epoch, with tests covering the mismatch.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems, stream-processing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100