jankotek / jankotek/mapdb

NPE in ByteBufferVol which appears to corrupt header

Open
#1,007 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
5.1k
Forks
877
PR merge metrics
No merged PRs in 30d

Description

We got this exception:

java.lang.NullPointerException
at org.mapdb.volume.ByteBufferVol.getSlice(ByteBufferVol.java:42)
at org.mapdb.volume.ByteBufferVol.getLong(ByteBufferVol.java:121)
at org.mapdb.StoreDirect.longStackTake(StoreDirect.kt:389)
at org.mapdb.StoreDirectAbstract.allocateData(StoreDirectAbstract.kt:293)
at org.mapdb.StoreDirect.put(StoreDirect.kt:618)
at org.mapdb.HTreeMap.valueWrap(HTreeMap.kt:1208)
at org.mapdb.HTreeMap.putprotected(HTreeMap.kt:344)
at org.mapdb.HTreeMap.put(HTreeMap.kt:324)

This is similar to the exception reported in #963 but not identical.

This appears to have corrupted the MapDB header as upon restart we got

java[11865]: at org.mapdb.DBMaker$Maker.make(DBMaker.kt:450)
java[11865]: at org.mapdb.StoreDirect$Companion.make$default(StoreDirect.kt:56)
java[11865]: at org.mapdb.StoreDirect$Companion.make(StoreDirect.kt:57)
java[11865]: at org.mapdb.StoreDirect.(StoreDirect.kt:114)
java[11865]: at org.mapdb.StoreDirectAbstract.fileHeaderCheck(StoreDirectAbstract.kt:113)
java[11865]: Exception in thread "main" org.mapdb.DBException$DataCorruption: Header checksum broken. Sto

(This is inverted because it's from a systemctl log.)

We are using MapDB 3.0.8 and it is instantiated like so:

        mapDB = DBMaker
            .fileDB(mapDBFile)
            .closeOnJvmShutdown()
            .fileMmapEnable()
            .fileMmapPreclearDisable()
            .make();

Transactions are not enabled because we need to be able to shrink the file. However, we

  1. Always call commit() after changing the map (there is only one map).
  2. Always change the map inside a Guava Monitor so two threads cannot change it at the same time.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting org.mapdb.volume.ByteBufferVol.getSlice at ByteBufferVol.java:42 and its caller getLong, then follow the StoreDirect allocation path shown in the trace. Reproduce with MapDB 3.0.8 using fileDB, fileMmapEnable, and fileMmapPreclearDisable if possible. Done means identifying the cause of the NPE and resulting header checksum corruption, with a regression test or documented reproduction.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kotlin
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.