JanusGraph / JanusGraph/janusgraph
Traversal skip records when multiply has steps is used by indexed fields.
- Dominant language
- Java
- Stars
- 5.8k
- Forks
- 1.2k
- Avg merge
- 13h 53m
- Merged PRs (30d)
- 6
Description
Hi all
I have found a problem:
I have graph with 100k + nodes.
```
g.traversal().V().has("graphId", P.within(UUID.fromString("ff28ef98-3bbd-4dd0-886f-6680445fc0fc"))).has("__tenant_key", P.eq("test")).hasLabel("Software").has("name", "vim").valueMap("name", "version", "kind")
```
Result:
```
==>[kind:[Software],name:[vim],version:[5.3]]
==>[kind:[Software],name:[vim],version:[5.2]]
==>[kind:[Software],name:[vim],version:[6.0]]
==>[kind:[Software],name:[vim],version:[5.1]]
==>[kind:[Software],name:[vim],version:[5.8]]
==>[kind:[Software],name:[vim],version:[6.2]]
==>[kind:[Software],name:[vim],version:[4.0]]
==>[kind:[Software],name:[vim],version:[7.2]]
==>[kind:[Software],name:[vim],version:[5.5]]
==>[kind:[Software],name:[vim],version:[7.1.314]]
==>[kind:[Software],name:[vim],version:[5.4]]
==>[kind:[Software],name:[vim],version:[5.7]]
==>[kind:[Software],name:[vim],version:[8.0.0377]]
==>[kind:[Software],name:[vim],version:[8.0.0376]]
==>[kind:[Software],name:[vim],version:[6.3.025]]
==>[kind:[Software],name:[vim],version:[8.0.0055]]
==>[kind:[Software],name:[vim],version:[8.0.1187]]
==>[kind:[Software],name:[vim],version:[6.3.030]]
==>[kind:[Software],name:[vim],version:[6.3.044]]
==>[kind:[Software],name:[vim],version:[6.3.011]]
==>[kind:[Software],name:[vim],version:[6.3]]
==>[kind:[Software],name:[vim],version:[6.4]]
==>[kind:[Software],name:[vim],version:[1.0]]
==>[kind:[Software],name:[vim],version:[3.0]]
==>[kind:[Software],name:[vim],version:[7.2a.10]]
==>[kind:[Software],name:[vim],version:[7.1.266]]
==>[kind:[Software],name:[vim],version:[8.0]]
==>[kind:[Software],name:[vim],version:[1.22]]
==>[kind:[Software],name:[vim],version:[5.6]]
==>[kind:[Software],name:[vim],version:[5.0]]
==>[kind:[Software],name:[vim],version:[6.1]]
==>[kind:[Software],name:[vim],version:[6.3.081]]
==>[kind:[Software],name:[vim],version:[7.1]]
==>[kind:[Software],name:[vim],version:[7.1.38]]
==>[kind:[Software],name:[vim],version:[7.0]]
```
Profile:
```
==>Traversal Metrics
Step Count Traversers Time (ms) % Dur
=============================================================================================================
JanusGraphStep([],[graphId.within([ff28ef98-3bb... 35 35 8.397 79.73
\_condition=(graphId = ff28ef98-3bbd-4dd0-886f-6680445fc0fc AND __tenant_key = test AND ~label = Software
AND name = vim)
\_isFitted=false
\_query=multiKSQ[1]@2147483647
\_index=byName
\_orders=[]
\_isOrdered=true
optimization 0.378
PropertyMapStep([name, version, kind],value) 35 35 2.134 20.27
>TOTAL - - 10.532 -
```
Looks good!
But if I add additional `has`
```
gremlin> g.traversal().V().has("graphId", P.within(UUID.fromString("ff28ef98-3bbd-4dd0-886f-6680445fc0fc"))).has("__tenant_key", P.eq("test")).hasLabel("Software").has("kind", "Software").has("name", "vim").valueMap("name", "version", "kind")
```
Result is
```
==>[kind:[Software],name:[vim],version:[5.3]]
==>[kind:[Software],name:[vim],version:[5.2]]
==>[kind:[Software],name:[vim],version:[6.0]]
==>[kind:[Software],name:[vim],version:[5.1]]
```
only 4 records.
Profile
```
==>Traversal Metrics
Step Count Traversers Time (ms) % Dur
=============================================================================================================
JanusGraphStep([],[graphId.within([ff28ef98-3bb... 4 4 1551.872 99.96
\_condition=(graphId = ff28ef98-3bbd-4dd0-886f-6680445fc0fc AND __tenant_key = test AND ~label = Software
AND kind = Software AND name = vim)
\_isFitted=false
\_query=multiKSQ[1]@2147483647
\_index=byName
\_orders=[]
\_isOrdered=true
optimization 1.512
backend-query 35 1.959
\_query=byName:multiKSQ[1]@16000
\_limit=16000
backend-query 16000 1548.215
\_query=byKind:multiKSQ[1]@2147483647
PropertyMapStep([name, version, kind],value) 4 4 0.646 0.04
>TOTAL - - 1552.518 -
```
I think problem with `_limit=16000`, but I can't found any information about this limit.
Thank you!
Contributor guide
Research direction
Start with the reported JanusGraphStep traversal profiles and compare the backend-query entries for the byName and byKind indexes, especially the _limit=16000 behavior. Reproduce both traversals from the issue against a graph containing more than 35 matching records, then verify that adding the kind predicate does not omit valid records and that the traversal returns all expected matches.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100