Lucene index is updated incorrectly during some dataset changes
- Dominant language
- Java
- Stars
- 1.4k
- Forks
- 712
- Avg merge
- 15h 41m
- Merged PRs (30d)
- 53
Description
### Version
4.9.0
### What happened?
Hi folks!
I'd like to offer text search on one of my Fuseki read-write datasets and noticed some irregularities when changing triples in the underlying dataset.
Setup:
- [Fuseki Standalone Server](https://repo1.maven.org/maven2/org/apache/jena/apache-jena-fuseki/4.9.0/)
- config: [config-text-tdb2.ttl](https://github.com/apache/jena/blob/main/jena-fuseki2/examples/config-text-tdb2.ttl)
- start Fuseki: `./fuseki-server --config=config-text-tdb2.ttl`
- data: [dfgfo.ttl](https://github.com/tibonto/DFG-Fachsystematik-Ontology/blob/main/dfgfo.ttl)
Query:
```
PREFIX rdfs:
PREFIX text:
SELECT ?uri ?score
WHERE {
(?uri ?score) text:query (rdfs:label 'ancient history') .
}
ORDER BY DESC(?score)
```
1. Add dfgfo.ttl to the dataset via *add data* in the UI (`POST http://127.0.0.1:3030/dataset/data`); we have 1126 triples
2. Execute query: 14 results
3. Repeat 1.
4. Execute query: 14 results (OK!)
5. Edit the dataset in the UI and save it without changes (`PUT http://127.0.0.1:3030/dataset/data?graph=default`); we still have 1126 triples
6. Execute query: 28 results (14 duplicates)
7. Send `DROP ALL` to `/dataset/update`; 0 triples now
8. Execute query: 28 results
9. Repeat 1.; we have 1126 triples again
10. Execute query: 42 results (14+14 duplicates)
### Relevant output and stacktrace
_No response_
### Are you interested in making a pull request?
None
Contributor guide
Research direction
Start by reproducing the sequence with examples/config-text-tdb2.ttl: load dfgfo.ttl, run the text query, save the unchanged dataset, issue DROP ALL, and reload it. The fix is done when dataset edits, clearing the dataset, and reloading do not leave duplicate Lucene search results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100