apache / apache/jena

Lucene index is updated incorrectly during some dataset changes

Open
#1,985 7 comments 0 reactions 0 assignees View on GitHub
bug Text Index
Dominant language
Java
Stars
1.4k
Forks
712
Avg merge
15h 41m
Merged PRs (30d)
53

Description

### Version

4.9.0

### What happened?

Hi folks!

I'd like to offer text search on one of my Fuseki read-write datasets and noticed some irregularities when changing triples in the underlying dataset.

Setup:
- [Fuseki Standalone Server](https://repo1.maven.org/maven2/org/apache/jena/apache-jena-fuseki/4.9.0/)
- config: [config-text-tdb2.ttl](https://github.com/apache/jena/blob/main/jena-fuseki2/examples/config-text-tdb2.ttl)
- start Fuseki: `./fuseki-server --config=config-text-tdb2.ttl`
- data: [dfgfo.ttl](https://github.com/tibonto/DFG-Fachsystematik-Ontology/blob/main/dfgfo.ttl)

Query:
```
PREFIX rdfs:
PREFIX text:

SELECT ?uri ?score
WHERE {
(?uri ?score) text:query (rdfs:label 'ancient history') .
}
ORDER BY DESC(?score)
```

1. Add dfgfo.ttl to the dataset via *add data* in the UI (`POST http://127.0.0.1:3030/dataset/data`); we have 1126 triples
2. Execute query: 14 results
3. Repeat 1.
4. Execute query: 14 results (OK!)
5. Edit the dataset in the UI and save it without changes (`PUT http://127.0.0.1:3030/dataset/data?graph=default`); we still have 1126 triples
6. Execute query: 28 results (14 duplicates)
7. Send `DROP ALL` to `/dataset/update`; 0 triples now
8. Execute query: 28 results
9. Repeat 1.; we have 1126 triples again
10. Execute query: 42 results (14+14 duplicates)

### Relevant output and stacktrace

_No response_

### Are you interested in making a pull request?

None

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the sequence with examples/config-text-tdb2.ttl: load dfgfo.ttl, run the text query, save the unchanged dataset, issue DROP ALL, and reload it. The fix is done when dataset edits, clearing the dataset, and reloading do not leave duplicate Lucene search results.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.