JanusGraph / JanusGraph/janusgraph

Traversal skip records when multiply has steps is used by indexed fields.

Open
#918 7 comments 0 reactions 0 assignees View on GitHub
kind/bug/possible
Dominant language
Java
Stars
5.8k
Forks
1.2k
Avg merge
13h 53m
Merged PRs (30d)
6

Description

Hi all
I have found a problem:
I have graph with 100k + nodes.

```
g.traversal().V().has("graphId", P.within(UUID.fromString("ff28ef98-3bbd-4dd0-886f-6680445fc0fc"))).has("__tenant_key", P.eq("test")).hasLabel("Software").has("name", "vim").valueMap("name", "version", "kind")
```
Result:
```
==>[kind:[Software],name:[vim],version:[5.3]]
==>[kind:[Software],name:[vim],version:[5.2]]
==>[kind:[Software],name:[vim],version:[6.0]]
==>[kind:[Software],name:[vim],version:[5.1]]
==>[kind:[Software],name:[vim],version:[5.8]]
==>[kind:[Software],name:[vim],version:[6.2]]
==>[kind:[Software],name:[vim],version:[4.0]]
==>[kind:[Software],name:[vim],version:[7.2]]
==>[kind:[Software],name:[vim],version:[5.5]]
==>[kind:[Software],name:[vim],version:[7.1.314]]
==>[kind:[Software],name:[vim],version:[5.4]]
==>[kind:[Software],name:[vim],version:[5.7]]
==>[kind:[Software],name:[vim],version:[8.0.0377]]
==>[kind:[Software],name:[vim],version:[8.0.0376]]
==>[kind:[Software],name:[vim],version:[6.3.025]]
==>[kind:[Software],name:[vim],version:[8.0.0055]]
==>[kind:[Software],name:[vim],version:[8.0.1187]]
==>[kind:[Software],name:[vim],version:[6.3.030]]
==>[kind:[Software],name:[vim],version:[6.3.044]]
==>[kind:[Software],name:[vim],version:[6.3.011]]
==>[kind:[Software],name:[vim],version:[6.3]]
==>[kind:[Software],name:[vim],version:[6.4]]
==>[kind:[Software],name:[vim],version:[1.0]]
==>[kind:[Software],name:[vim],version:[3.0]]
==>[kind:[Software],name:[vim],version:[7.2a.10]]
==>[kind:[Software],name:[vim],version:[7.1.266]]
==>[kind:[Software],name:[vim],version:[8.0]]
==>[kind:[Software],name:[vim],version:[1.22]]
==>[kind:[Software],name:[vim],version:[5.6]]
==>[kind:[Software],name:[vim],version:[5.0]]
==>[kind:[Software],name:[vim],version:[6.1]]
==>[kind:[Software],name:[vim],version:[6.3.081]]
==>[kind:[Software],name:[vim],version:[7.1]]
==>[kind:[Software],name:[vim],version:[7.1.38]]
==>[kind:[Software],name:[vim],version:[7.0]]
```
Profile:
```
==>Traversal Metrics
Step Count Traversers Time (ms) % Dur
=============================================================================================================
JanusGraphStep([],[graphId.within([ff28ef98-3bb... 35 35 8.397 79.73
\_condition=(graphId = ff28ef98-3bbd-4dd0-886f-6680445fc0fc AND __tenant_key = test AND ~label = Software
AND name = vim)
\_isFitted=false
\_query=multiKSQ[1]@2147483647
\_index=byName
\_orders=[]
\_isOrdered=true
optimization 0.378
PropertyMapStep([name, version, kind],value) 35 35 2.134 20.27
>TOTAL - - 10.532 -
```
Looks good!

But if I add additional `has`

```
gremlin> g.traversal().V().has("graphId", P.within(UUID.fromString("ff28ef98-3bbd-4dd0-886f-6680445fc0fc"))).has("__tenant_key", P.eq("test")).hasLabel("Software").has("kind", "Software").has("name", "vim").valueMap("name", "version", "kind")
```
Result is
```
==>[kind:[Software],name:[vim],version:[5.3]]
==>[kind:[Software],name:[vim],version:[5.2]]
==>[kind:[Software],name:[vim],version:[6.0]]
==>[kind:[Software],name:[vim],version:[5.1]]
```
only 4 records.

Profile
```
==>Traversal Metrics
Step Count Traversers Time (ms) % Dur
=============================================================================================================
JanusGraphStep([],[graphId.within([ff28ef98-3bb... 4 4 1551.872 99.96
\_condition=(graphId = ff28ef98-3bbd-4dd0-886f-6680445fc0fc AND __tenant_key = test AND ~label = Software
AND kind = Software AND name = vim)
\_isFitted=false
\_query=multiKSQ[1]@2147483647
\_index=byName
\_orders=[]
\_isOrdered=true
optimization 1.512
backend-query 35 1.959
\_query=byName:multiKSQ[1]@16000
\_limit=16000
backend-query 16000 1548.215
\_query=byKind:multiKSQ[1]@2147483647
PropertyMapStep([name, version, kind],value) 4 4 0.646 0.04
>TOTAL - - 1552.518 -
```
I think problem with `_limit=16000`, but I can't found any information about this limit.

Thank you!

Contributor guide

Open the contributing guide

Research direction

Start with the reported JanusGraphStep traversal profiles and compare the backend-query entries for the byName and byKind indexes, especially the _limit=16000 behavior. Reproduce both traversals from the issue against a graph containing more than 35 matching records, then verify that adding the kind predicate does not omit valid records and that the traversal returns all expected matches.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.