JanusGraph / JanusGraph/janusgraph

StandardScannerExecutor will stuck on ghost vertices

Open
#1,750 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/core
Dominant language
Java
Stars
5.8k
Forks
1.2k
Avg merge
13h 53m
Merged PRs (30d)
6

Description

It seems second condition check for vertex existing - not ghost vertex
```
if (currentResults[i]!=null && currentResults[i].key.equals(key)) {
assert query.equals(currentResults[i].query);
entries = currentResults[i].entries;
currentResults[i]=null;
}
```
https://github.com/JanusGraph/janusgraph/blob/master/janusgraph-core/src/main/java/org/janusgraph/diskstorage/keycolumnvalue/scan/StandardScannerExecutor.java#L167

But if is it ghost `currentResult[i]` stay filled and it lead to queue exhaust

```
for (int i = 0; i < numQueries; i++) {
if (currentResults[i]!=null) continue;
```

It lead to any scanner job including `IndexRepairJob`, `IndexRemoveJob`, `VertexProgramScanJob` may partially work. Issue https://github.com/JanusGraph/janusgraph/issues/1389 partially related too

GhostRemoveJob can help if will run before any scanner job, but can't due issue https://github.com/JanusGraph/janusgraph/issues/1749

I think second way to fix that issue just skip ghost vertice during scan

https://github.com/JanusGraph/janusgraph/blob/master/janusgraph-core/src/main/java/org/janusgraph/diskstorage/keycolumnvalue/scan/StandardScannerExecutor.java#L167
```
if (currentResults[i]!=null && currentResults[i].key.equals(key)) {
assert query.equals(currentResults[i].query);
entries = currentResults[i].entries;
}
currentResults[i] = null;
```

Workaround, remove ghost vertex before start main job

```
JanusGraphManagement.IndexJobFuture ghostRemover =
graph.getBackend().buildEdgeScanJob()
.setJob(new GhostVertexRemover(graph))
.execute();
try {
logger.info("GhostVertexRemover statistics: {},{},{}",
ghostRemover.get().getCustom(REMOVED_VERTEX_COUNT),
ghostRemover.get().getCustom(REMOVED_RELATION_COUNT),
ghostRemover.get().getCustom(SKIPPED_GHOST_LIMIT_COUNT));
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
```

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in janusgraph-core/src/main/java/org/janusgraph/diskstorage/keycolumnvalue/scan/StandardScannerExecutor.java around line 167 and trace how ghost vertices leave currentResults occupied. Review the affected scanner jobs, including IndexRepairJob, IndexRemoveJob, and VertexProgramScanJob, and confirm that scanning no longer exhausts its queue or partially processes work when ghost vertices are present.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.