Recovery failure when the tablet server shuts down due to 'No space left on device'
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Fluss version
0.9.0 (latest release)
### Please describe the bug 🐞
I found that recovery fails when the tablet server shuts down due to a 'No space left on device' error. The failure logs are as follows:
However, after dropping the corresponding partitions by fluss admin, partition recovery is still not skipped. This indicates that the behavior described in #3223 can be further improved.
In addition, we should prevent writes once the disk usage reaches a high-capacity watermark to avoid potential data loss.
### Solution
_No response_
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Review the recovery behavior described in issue #3223 and the tablet-server shutdown logs for the disk-full case. Verify why dropping the corresponding partitions through Fluss admin does not skip recovery, and define completion as correct recovery handling plus writes being prevented at the high-capacity watermark.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100