apache / apache/bookkeeper

SortedLedgerStorage creates too many entrylog files

Open
#3,743 5 comments 0 reactions 0 assignees View on GitHub
type/bug
Dominant language
Java
Stars
2k
Forks
976
Avg merge
6d 15h
Merged PRs (30d)
7

Description

*BUG REPORT*

***Describe the bug***

Recently our production bookie encounters 'too many open files' error, we found bookie create much more entrylog file files than we expected, and the metric bookie.bookie_gc_ACTIVE_ENTRY_LOG_COUNT_gauge like this,

image

There are many files which is very small and not satisfy 1GB size.

image

we use the **SortedLedgerStorage** and entryLogPerLedgerEnabled is **false**.
we use arthas to track the create entrylog logic and found this:

![image](https://user-images.githubusercontent.com/32755366/212841068-58b9540b-08a3-44ad-8f81-e8a2b11c1449.png)

It seems InterleavedLedgerStorage will re-startcheckpoint during rotate entrylog, this may cause memtable flush some bytes during perivous and this checkpoin, and createNewlog is invoked again, this situation will keep loop and cannot be broken.

image
image

Please confirm is it a bug or not, if you need more information, please tell me.
Thanks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.