apache / apache/bookkeeper

BookKeeper switched to readonly after FileInfoDeletedException

Open
#1,919 12 comments 0 reactions 0 assignees View on GitHub
area/bookie triage/week-7 type/bug
Dominant language
Java
Stars
2k
Forks
976
Avg merge
6d 15h
Merged PRs (30d)
7

Description

**BUG REPORT**

1. Please describe the issue you observed:
During normal operation of a cluster of 3 BookKeepers one switched to READONLY mode. Usually we see this kind of errors when a disk become full, but this time we found out a "FileInfoDeletedException" inside the logs. We restarted the Bookie and everything returned to normal.

Apache BookKeeper 4.7.3
Java 11.0.2+7

- What did you do?
n/a

- What did you expect to see?
no error/no readonly mode

- What did you see instead?
The Bookie switched to readonly mode
See stacktrace inside org.apache.bookkeeper.bookie.SortedLedgerStorage

`
19-01-30-09-48-40 org.apache.bookkeeper.bookie.FileInfo$FileInfoDeletedException: FileInfo already deleted
org.apache.bookkeeper.bookie.FileInfo$FileInfoDeletedException: FileInfo already deleted
at org.apache.bookkeeper.bookie.FileInfo.checkOpen(FileInfo.java:248)
at org.apache.bookkeeper.bookie.FileInfo.checkOpen(FileInfo.java:242)
at org.apache.bookkeeper.bookie.FileInfo.size(FileInfo.java:342)
at org.apache.bookkeeper.bookie.IndexPersistenceMgr.updatePage(IndexPersistenceMgr.java:643)
at org.apache.bookkeeper.bookie.IndexInMemPageMgr.grabLedgerEntryPage(IndexInMemPageMgr.java:470)
at org.apache.bookkeeper.bookie.IndexInMemPageMgr.getLedgerEntryPage(IndexInMemPageMgr.java:435)
at org.apache.bookkeeper.bookie.IndexInMemPageMgr.putEntryOffset(IndexInMemPageMgr.java:594)
at org.apache.bookkeeper.bookie.LedgerCacheImpl.putEntryOffset(LedgerCacheImpl.java:96)
at org.apache.bookkeeper.bookie.InterleavedLedgerStorage.processEntry(InterleavedLedgerStorage.java:433)
at org.apache.bookkeeper.bookie.SortedLedgerStorage.process(SortedLedgerStorage.java:184)
at org.apache.bookkeeper.bookie.EntryMemTable.flushSnapshot(EntryMemTable.java:251)
at org.apache.bookkeeper.bookie.EntryMemTable.flush(EntryMemTable.java:205)
at org.apache.bookkeeper.bookie.SortedLedgerStorage$1.run(SortedLedgerStorage.java:213)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.