Transaction DB holds too much memory
- Dominant language
- C++
- Stars
- 32.1k
- Forks
- 6.9k
- Avg merge
- 32m
- Merged PRs (30d)
- 1
Description
> Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://www.facebook.com/groups/rocksdb.dev
We have a scenario needs multiple DB instances and also multiple CFs within a DB with transaction support.
Transaction DB keeps **max_write_buffer_number** of memtable even when it is flushed, considering the number of DB instance times the CF count, it's significant of memory consumption.
I know those memtable is held for checking write conflicts, but it doesn't seem to be freed when there is no write for a while.
The work around for us is setting **max_write_buffer_number_to_maintain = 1,** but even so, it still uses a lot of memory.
### Expected behavior
Tx DB should free memtable when there is no write for a while, or at least can be freed by manual flush, so the user can decide to free up the memory when there is no recent write.
### Actual behavior
Tx DB consistently holds max_write_buffer_number_to_maintain of memtables, even there is no recent write. Manual flush doesn't help either.
### Steps to reproduce the behavior
Create TransactionDB with max_write_buffer_number = 3, write_buffer_size=64M. Then write enough data (more than 3 * 64MB). DB holds ~192MB memory forever, even after manual flush. If there is N CF in the DB, then the memory overhead becomes even more significant.
Cc: @yuslepukhin
Contributor guide
Assessment
This issue has not been assessed yet.