SPARK: current database instance is not active

OrientDB Version: 3.0.34

Java Version: 1.8

spark : local[*]

OS: Red Hat Enterprise Linux Server 7.1 (Maipo)

Context

• I am running a spark(2.3.4) application to populate an orientdb (3.0.34) database.
• OrientDB And ODatabasePool are initialized on the Driver program

     this.context = new OrientDB(remoteConnexionCommand, dbUser, dbPwd, dbConfig)
     this.pool = new ODatabasePool(this.context, globalConfig.getString("db.name"), dbUser, dbPwd)

Code to create a connection to db is executed inside a rdd.foreach method as described here:

     rdd.repartition(20).foreach(row => {
       try {
           DBManager session = DBManager.getSession() // which init a new Session given an initialized pool
           session.command(statement, params)
        } catch {
           case t: Throwable => ...
        }

The job is launched via the foreach action and the getSession code goes like this:

       def getSession(): DBSession= {
            if (this.pool == null) throw new Exception("") else new DBSession( this.pool.acquire())
        }

Also at the end of each query we call a

session.activateOnCurrentThread()

Expected behavior

No error

Actual behavior

Given this code I get the following error wihch appear to happen in an unpredicted manner:

The current database instance (com.orientechnologies.orient.core.db.ODatabaseDocumentRemotePooled@737aa62f) is not active on the current thread (Thread[Executor task launch worker for task 319,5,main]). Current active database is: com.orientechnologies.orient.core.db.ODatabaseDocumentRemotePooled@265d0416

Steps to reproduce

difficult to reproduce, working on it but the behavior is unexpected, and occurs randomly. Would like to know what would cause this kind of problems in the context of a spark app.

Start with the Spark rdd.foreach flow and the DBManager.getSession, ODatabasePool, DBSession, and activateOnCurrentThread calls shown in the report. Reproduce with OrientDB 3.0.34, Java 1.8, Spark 2.3.4, and local[*], then trace which pooled database instance is active on each executor thread. Done means identifying the cause of the intermittent mismatch and documenting or testing a reproducible resolution.