JanusGraph / JanusGraph/janusgraph

Index issue when a value is removed and added to a list property

Open
#3,734 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Java
Stars
5.8k
Forks
1.2k
Avg merge
13h 53m
Merged PRs (30d)
6

Description

Hello,

I found a bug on the indexation of list property. When the same value is removed and added again to the list in the same transaction, the value is duplicated in the index. I reproduced the problem with Cassandra/ES and Berkeley/Lucene.

The database schema creation

```groovy
graph = JanusGraphFactory.open('conf/janusgraph-berkeletje-lucene.properties')

// Create schema
mgmt = graph.openManagement()
tags = mgmt.makePropertyKey('tags').dataType(String.class).cardinality(Cardinality.LIST).make()
data = mgmt.makeVertexLabel('data').make()
mgmt.commit()

// Add indexes
mgmt = graph.openManagement()
tags = mgmt.getPropertyKey('tags')
mgmt.buildIndex('global', Vertex.class).addKey(tags, Mapping.STRING.asParameter()).buildMixedIndex('search')
mgmt.commit()

// Wait the indexes
ManagementSystem.awaitGraphIndexStatus(graph, 'global').call()

// Reindex data
mgmt = graph.openManagement()
mgmt.updateIndex(mgmt.getGraphIndex('global'), SchemaAction.REINDEX).get()
mgmt.commit()

// Wait the indexes
ManagementSystem.awaitGraphIndexStatus(graph, 'global').status(SchemaStatus.ENABLED).call()
```

The code below describes how to reproduce:

```groovy
g = graph.traversal()

v = g.addV('data').property(list, 'tags', 't1').id().next()
g.tx().commit() // index contains tags:[t1]
g.V(c).valueMap() // ==>[tags:[t1]]

g.V(v).properties('tags').drop().iterate()
g.V(v).property(list, 'tags', 't1').iterate()
g.tx().commit() // index contains tags:[t1, t1]
g.V(c).valueMap() // ==>[tags:[t1]]

g.V(v).properties('tags').drop().iterate()
g.V(v).property(list, 'tags', 't2').iterate()
g.tx().commit() // index contains tags:[t1, t2]
g.V(c).valueMap() // ==>[tags:[t2]]

g.V().has('tags', 't1') // should return empty traversal but returns the vertex v
```

I used to drop all values of the list and add new values because I want to set the entire list values in only one traversal (using sideEffect). I don't know if there is another way to do that.
The problem doesn't occur with the "set" properties. Is it possible to convert a list property into set property without copying all values?

Contributor guide

Open the contributing guide

Research direction

Start by running the supplied Groovy reproduction with the Cassandra/Elasticsearch and Berkeley/Lucene configurations, focusing on list-property removal and re-addition in one transaction. Trace the mixed-index update path used by the property drop/add operations. Done means the index has no duplicate or stale values and the query for removed value t1 returns no vertex.

Written by the indexing model from the issue text.

Assessment

Tech stack
cassandra, elasticsearch, groovy, java
Domain
databases, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.