[server] KV writes stall indefinitely when a WAL batch exhausts the shared memory pool
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Fluss version
0.9.0 (latest release)
### Please describe the bug 🐞
A KV write batch can generate WAL larger than the shared memory pool capacity, for example when small updates produce large full-row changelog images.
The batch incrementally acquires memory pages. Once it holds the entire pool, requesting another page causes it to wait indefinitely: its existing pages can only be released after the batch completes or fails, but completion requires more memory.
The blocked request also holds a KV write lock and occupies an RPC worker, causing other requests to stall.
The existing check for an individual allocation exceeding the pool capacity does not cover this case, because each allocation requests only a small page.
### Solution
Fail immediately when the memory already held by a WAL batch plus its next allocation exceeds the pool's total capacity.
Ensure the failure releases the batch's memory and locks and preserves write-failure and retry correctness. Keep the existing waiting behavior for temporary contention between batches.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
No files or tests are named. Trace the server KV WAL batch allocation path and the existing individual-allocation capacity check, then reproduce a batch that exceeds the shared pool. Done means the batch fails immediately, releases memory and locks, preserves write-failure and retry correctness, and still waits during temporary contention between batches.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100