logfile growing without bound if we keep writing empty WriteBatch (1.20)
- Dominant language
- C++
- Stars
- 39.4k
- Forks
- 8.2k
- PR merge metrics
- No merged PRs in 30d
Description
Hi all,
I am using leveldb 1.20.0, and I found that if we keep writing empty WriteBatch. the logfile size will increase without bound.
```
#include
#include
#include
#include
#include
using std::cout;
using std::endl;
static void print_usage(const char *pname) {
cout << "Usage: " << pname << " " << endl;
}
int main(int argc, const char **argv) {
if (argc != 3) {
print_usage(argv[0]);
return -1;
}
const std::string &dbpath {argv[1]};
std::istringstream iss {argv[2]};
uint64_t cnt = 0;
if (!(iss >> cnt)) {
print_usage(argv[0]);
return -1;
}
leveldb::Options options;
options.create_if_missing = true;
options.write_buffer_size = 65536; // 64KB
leveldb::DB *db_handle {nullptr};
leveldb::Status status = leveldb::DB::Open(options, dbpath, &db_handle);
if (!status.ok()) {
cout << "Open db " << dbpath << " failed : " << status.ToString() << endl;
return -1;
}
leveldb::WriteOptions write_options;
cout << "Start to put " << cnt << " Empty KV" << endl;
for (size_t i = 0; i < cnt; i++) {
leveldb::WriteBatch wb;
leveldb::Status w_status = db_handle->Write(write_options, &wb);
if (!w_status.ok()) {
cout << "Write db failed : " << w_status.ToString() << endl;
return -1;
}
}
return 0;
}
```
```
[root@7d2b7faa12eb ~]# ./test_empty_wb_write ./empty_wb_1000 1000
Start to put 1000 Empty KV
[root@7d2b7faa12eb ~]# ./test_empty_wb_write ./empty_wb_1000000 1000000
Start to put 1000000 Empty KV
[root@7d2b7faa12eb ~]# ./test_empty_wb_write ./empty_wb_1000000000 1000000000
Start to put 1000000000 Empty KV
[root@7d2b7faa12eb ~]# ls -alh empty_wb_1000
total 40K
drwxr-xr-x 2 root root 4.0K Jun 3 05:34 .
dr-xr-x--- 6 root root 4.0K Jun 3 05:43 ..
-rw-r--r-- 1 root root 19K Jun 3 05:34 000003.log
-rw-r--r-- 1 root root 16 Jun 3 05:34 CURRENT
-rw-r--r-- 1 root root 0 Jun 3 05:34 LOCK
-rw-r--r-- 1 root root 57 Jun 3 05:34 LOG
-rw-r--r-- 1 root root 50 Jun 3 05:34 MANIFEST-000002
[root@7d2b7faa12eb ~]# ls -alh empty_wb_1000000
total 19M
drwxr-xr-x 2 root root 4.0K Jun 3 05:34 .
dr-xr-x--- 6 root root 4.0K Jun 3 05:43 ..
-rw-r--r-- 1 root root 19M Jun 3 05:34 000003.log
-rw-r--r-- 1 root root 16 Jun 3 05:34 CURRENT
-rw-r--r-- 1 root root 0 Jun 3 05:34 LOCK
-rw-r--r-- 1 root root 57 Jun 3 05:34 LOG
-rw-r--r-- 1 root root 50 Jun 3 05:34 MANIFEST-000002
[root@7d2b7faa12eb ~]# ls -alh empty_wb_100000000
total 1.8G
drwxr-xr-x 2 root root 4.0K Jun 3 05:37 .
dr-xr-x--- 6 root root 4.0K Jun 3 05:43 ..
-rw-r--r-- 1 root root 1.8G Jun 3 05:42 000003.log
-rw-r--r-- 1 root root 16 Jun 3 05:37 CURRENT
-rw-r--r-- 1 root root 0 Jun 3 05:37 LOCK
-rw-r--r-- 1 root root 57 Jun 3 05:37 LOG
-rw-r--r-- 1 root root 50 Jun 3 05:37 MANIFEST-000002
```
I think is caused by memtable size not triggering the minor compaction , but keep appending log to logfile.
Is this a known and expected behavior, or is it already fixed in the later leveldb version?
Contributor guide
Assessment
This issue has not been assessed yet.