jankotek / jankotek/mapdb

Write hangs after earlier interrupted write

Open
#352 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

4.0 document
Dominant language
Java
Stars
5.1k
Forks
877
PR merge metrics
No merged PRs in 30d

Description

I have problems with writes sometimes hang after an earlier write has been interrupted (the thread doing the write got a interrupt).

Flow:

  1. Writer thread writes to MapDB
  2. Writer thread gets a interrupt and exits with java.io.IOError: java.nio.channels.ClosedByInterruptException
  3. I spawn a new writer thread
  4. New writer thread hangs on write (not throwing any IOError)

When I check the stack-trace of the hanged new writer thread it looks like this:

    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:349)
    at org.mapdb.BTreeMap.lock(BTreeMap.java:2882)
    at org.mapdb.BTreeMap.put2(BTreeMap.java:686)
    at org.mapdb.BTreeMap.put(BTreeMap.java:643)
    at {MyWriterThread}

It seems like there is some lock that isn't freed from last write error.
I know that further writes probably wouldn't succeed since underlying NIO channel is closed (due to interrupt) but the write shouldn't hang or?

This is the stack-trace for the earlier write that failed because of interrupt:

java.io.IOError: java.nio.channels.ClosedByInterruptException
    at org.mapdb.Volume$FileChannelVol.putByte(Volume.java:761)
    at org.mapdb.StoreWAL.walIndexVal(StoreWAL.java:309)
    at org.mapdb.StoreWAL.update(StoreWAL.java:445)
    at org.mapdb.EngineWrapper.update(EngineWrapper.java:63)
    at org.mapdb.BTreeMap.put2(BTreeMap.java:746)
    at org.mapdb.BTreeMap.put(BTreeMap.java:643)
    at {MyWriterThread} 
Caused by: java.nio.channels.ClosedByInterruptException
    at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
    at sun.nio.ch.FileChannelImpl.writeInternal(FileChannelImpl.java:743)
    at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:723)
    at org.mapdb.Volume$FileChannelVol.writeFully(Volume.java:706)
    at org.mapdb.Volume$FileChannelVol.putByte(Volume.java:759)
    ... 8 more

It is not always happening so it seems like timing / data-size dependent when this state occurs.

Something that got me wondering is these catches in the end of BTreeMap.put2():

        }catch(RuntimeException e){
            unlockAll(nodeLocks);
            throw e;
        }catch(Exception e){
            unlockAll(nodeLocks);
            throw new RuntimeException(e);
        }

They will not catch IOError from engine calls so maybe that has something to do with locks not being released?

I open the DB with: BMaker.newFileDB(file).compressionEnable().cacheDisable().make();

I haven't been able to write a short test program to reproduce it because the timing/data-size dependency.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with BTreeMap.put2 and its lock handling, then trace the interrupted write through StoreWAL.update and Volume$FileChannelVol.putByte. Try to reproduce the sequence of an interrupted writer followed by a new writer using the provided file-backed configuration, including timing and data-size variations. Done means a subsequent write does not hang and the interrupted state is handled consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.