JanusGraph / JanusGraph/janusgraph
Index issue when a value is removed and added to a list property
- Dominant language
- Java
- Stars
- 5.8k
- Forks
- 1.2k
- Avg merge
- 13h 53m
- Merged PRs (30d)
- 6
Description
Hello,
I found a bug on the indexation of list property. When the same value is removed and added again to the list in the same transaction, the value is duplicated in the index. I reproduced the problem with Cassandra/ES and Berkeley/Lucene.
The database schema creation
```groovy
graph = JanusGraphFactory.open('conf/janusgraph-berkeletje-lucene.properties')
// Create schema
mgmt = graph.openManagement()
tags = mgmt.makePropertyKey('tags').dataType(String.class).cardinality(Cardinality.LIST).make()
data = mgmt.makeVertexLabel('data').make()
mgmt.commit()
// Add indexes
mgmt = graph.openManagement()
tags = mgmt.getPropertyKey('tags')
mgmt.buildIndex('global', Vertex.class).addKey(tags, Mapping.STRING.asParameter()).buildMixedIndex('search')
mgmt.commit()
// Wait the indexes
ManagementSystem.awaitGraphIndexStatus(graph, 'global').call()
// Reindex data
mgmt = graph.openManagement()
mgmt.updateIndex(mgmt.getGraphIndex('global'), SchemaAction.REINDEX).get()
mgmt.commit()
// Wait the indexes
ManagementSystem.awaitGraphIndexStatus(graph, 'global').status(SchemaStatus.ENABLED).call()
```
The code below describes how to reproduce:
```groovy
g = graph.traversal()
v = g.addV('data').property(list, 'tags', 't1').id().next()
g.tx().commit() // index contains tags:[t1]
g.V(c).valueMap() // ==>[tags:[t1]]
g.V(v).properties('tags').drop().iterate()
g.V(v).property(list, 'tags', 't1').iterate()
g.tx().commit() // index contains tags:[t1, t1]
g.V(c).valueMap() // ==>[tags:[t1]]
g.V(v).properties('tags').drop().iterate()
g.V(v).property(list, 'tags', 't2').iterate()
g.tx().commit() // index contains tags:[t1, t2]
g.V(c).valueMap() // ==>[tags:[t2]]
g.V().has('tags', 't1') // should return empty traversal but returns the vertex v
```
I used to drop all values of the list and add new values because I want to set the entire list values in only one traversal (using sideEffect). I don't know if there is another way to do that.
The problem doesn't occur with the "set" properties. Is it possible to convert a list property into set property without copying all values?
Contributor guide
Research direction
Start by running the supplied Groovy reproduction with the Cassandra/Elasticsearch and Berkeley/Lucene configurations, focusing on list-property removal and re-addition in one transaction. Trace the mixed-index update path used by the property drop/add operations. Done means the index has no duplicate or stale values and the query for removed value t1 returns no vertex.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cassandra, elasticsearch, groovy, java
- Domain
- databases, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100