JanusGraph / JanusGraph/janusgraph
Index mutation failures don't cause transaction abort
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 5.8k
- Forks
- 1.2k
- Avg merge
- 13h 53m
- Merged PRs (30d)
- 6
Description
When a JanusGraph vertex property indexed in ElasticSearch is updated concurrently in separate transactions and then committed, ElasticSearch fails and the indexing backend throws an exception like so:
```ERROR 2017-05-24T07:20:39.703-07:00 [Grizzly(2)] org.janusgraph.diskstorage.es.ElasticSearchIndex - Failed to execute ES query 1m
java.lang.Exception: failure in bulk execution:
[0]: index [titan], type [verticesByDocumentExternal], id [j3s], message [VersionConflictEngineException[[titan][4] [verticesByDocumentExternal][j3s]: version conflict, current [3], provided [2]]]
```
That isn't a problem by itself. However, JanusGraph swallows this exception and proceeds to commit the transaction as if nothing had happened. See this snippet from `StandardJanusGraph` (I am on 0.1.1 but it appears the same on the master branch):
```
//2. Commit indexes - [FAILURE] all exceptions are collected and logged but nothing is aborted
indexFailures = mutator.commitIndexes();
if (!indexFailures.isEmpty()) {
status = LogTxStatus.SECONDARY_FAILURE;
for (Map.Entry entry : indexFailures.entrySet()) {
log.error("Error while commiting index mutations for transaction ["+transactionId+"] on index: " +entry.getKey(),entry.getValue());
}
}
//3. Log transaction if configured - [FAILURE] is recorded but does not cause exception
if (logTxIdentifier!=null) {
try {
userlogSuccess = false;
final Log userLog = backend.getUserLog(logTxIdentifier);
Future env = userLog.add(txLogHeader.serializeModifications(serializer, LogTxStatus.USER_LOG, tx, addedRelations, deletedRelations));
if (env.isDone()) {
try {
env.get();
} catch (ExecutionException ex) {
throw ex.getCause();
}
}
userlogSuccess=true;
} catch (Throwable e) {
status = LogTxStatus.SECONDARY_FAILURE;
log.error("Could not user-log committed transaction ["+transactionId+"] to " + logTxIdentifier, e);
}
}
```
When this happens, I would expect the commit to fail, and I'd be fine with that - essentially the same behavior as conflicting transactions in JanusGraph have in many circumstances. But since it doesn't, my transaction completes and I am left with some weird state, essentially a transaction that is partially committed.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the commit path in StandardJanusGraph, especially mutator.commitIndexes(), and inspect how ElasticSearchIndex reports failures. Determine how an index mutation failure should affect transaction status and verify that the transaction no longer completes as successfully when indexing fails.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, java
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100