JanusGraph / JanusGraph/janusgraph
StandardScannerExecutor will stuck on ghost vertices
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 5.8k
- Forks
- 1.2k
- Avg merge
- 13h 53m
- Merged PRs (30d)
- 6
Description
It seems second condition check for vertex existing - not ghost vertex
```
if (currentResults[i]!=null && currentResults[i].key.equals(key)) {
assert query.equals(currentResults[i].query);
entries = currentResults[i].entries;
currentResults[i]=null;
}
```
https://github.com/JanusGraph/janusgraph/blob/master/janusgraph-core/src/main/java/org/janusgraph/diskstorage/keycolumnvalue/scan/StandardScannerExecutor.java#L167
But if is it ghost `currentResult[i]` stay filled and it lead to queue exhaust
```
for (int i = 0; i < numQueries; i++) {
if (currentResults[i]!=null) continue;
```
It lead to any scanner job including `IndexRepairJob`, `IndexRemoveJob`, `VertexProgramScanJob` may partially work. Issue https://github.com/JanusGraph/janusgraph/issues/1389 partially related too
GhostRemoveJob can help if will run before any scanner job, but can't due issue https://github.com/JanusGraph/janusgraph/issues/1749
I think second way to fix that issue just skip ghost vertice during scan
https://github.com/JanusGraph/janusgraph/blob/master/janusgraph-core/src/main/java/org/janusgraph/diskstorage/keycolumnvalue/scan/StandardScannerExecutor.java#L167
```
if (currentResults[i]!=null && currentResults[i].key.equals(key)) {
assert query.equals(currentResults[i].query);
entries = currentResults[i].entries;
}
currentResults[i] = null;
```
Workaround, remove ghost vertex before start main job
```
JanusGraphManagement.IndexJobFuture ghostRemover =
graph.getBackend().buildEdgeScanJob()
.setJob(new GhostVertexRemover(graph))
.execute();
try {
logger.info("GhostVertexRemover statistics: {},{},{}",
ghostRemover.get().getCustom(REMOVED_VERTEX_COUNT),
ghostRemover.get().getCustom(REMOVED_RELATION_COUNT),
ghostRemover.get().getCustom(SKIPPED_GHOST_LIMIT_COUNT));
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
```
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in janusgraph-core/src/main/java/org/janusgraph/diskstorage/keycolumnvalue/scan/StandardScannerExecutor.java around line 167 and trace how ghost vertices leave currentResults occupied. Review the affected scanner jobs, including IndexRepairJob, IndexRemoveJob, and VertexProgramScanJob, and confirm that scanning no longer exhausts its queue or partially processes work when ghost vertices are present.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100