jankotek / jankotek/mapdb

concurrency reading problem ?

Open
#752 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

4.0 optimize
Dominant language
Java
Stars
5.1k
Forks
877
PR merge metrics
No merged PRs in 30d

Description

I have a task with a lot of lookups. They usually goes quite near in terms of keys.

The effect is I can not get more than about 5-6 cores working effectivly.

Other threads gets locked in the last line of code below (StoreDirectAbstract, lines 37+):

//TODO PERF indexPages are synchronized writes are protected by structural lock, but should it be read under locks?
protected val indexPages = if(isThreadSafe) LongArrayList().asSynchronized() else LongArrayList()

protected fun recidToOffset(recid2:Long):Long{
    var recid = recid2-1; //normalize recid so it starts from zero
    if(recid< StoreDirectJava.RECIDS_PER_ZERO_INDEX_PAGE){
        //zero index page
        return StoreDirectJava.HEAD_END + 16 + recid*8
    }
    //strip zero index page
    recid -= StoreDirectJava.RECIDS_PER_ZERO_INDEX_PAGE
    val pageNum = recid/ StoreDirectJava.RECIDS_PER_INDEX_PAGE
    return indexPages.get(pageNum.toInt()) + 16 + ((recid)% StoreDirectJava.RECIDS_PER_INDEX_PAGE)*8
}

seems that indexPages.get(pageNum.toInt()) causes the problem, as this call is synchronized.

below actual implementation of get (from eclipse collections):

public long get(int index)
{
    synchronized (this.getLock())
    {
        return this.getMutableLongList().get(index);
    }
}

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in StoreDirectAbstract at recidToOffset and inspect the indexPages access, then review the shown Eclipse Collections get implementation to understand the synchronization. Reproduce the lookup workload and determine whether indexPages.get causes the contention; done should preserve thread safety while addressing the reported concurrency problem.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.