Long lived Iterator pinning flushed MemTable and compacted SSTs
- Dominant language
- C++
- Stars
- 32.1k
- Forks
- 6.9k
- Avg merge
- 32m
- Merged PRs (30d)
- 1
Description
### Expected behavior
Flushed **MemTable**s and compacted **SST**s should be deleted.
### Actual behavior
Long lived Iterators referring `Version` objects which referring **MemTable**s and compacted **SST**s which lead to memory over consumption while writing to DB.
### Steps to reproduce the behavior
Create an iterator and using it while writing to DB, newly created MemTables can not be freed.
------
We noticed this issue when we running sysbench on myrocks, we also have filed an [issue](https://github.com/facebook/mysql-5.6/issues/1211) and [PR](https://github.com/facebook/mysql-5.6/pull/1212) for MyRocks, we worked around this issue by creating a new Iterator and delete old Iterator periodically. It is better to resolve this issue in RocksDB.
Contributor guide
Research direction
The report names long-lived Iterators, Version objects, MemTables, and compacted SSTs but does not identify files or tests. Start by reproducing the issue while writing to the database with an iterator held open, then trace the referenced object lifetimes and add or run a regression test showing that obsolete flushed MemTables and compacted SSTs are released without requiring periodic iterator replacement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100