dice-group / dice-group/gerbil
SameAs retrieval causes problems in a JUnit test
- Dominant language
- Java
- Stars
- 231
- Forks
- 55
- PR merge metrics
- No merged PRs in 30d
Description
## Description
The D2KBTest fails because two entities of the OKE Task 1 Example Dataset seem to be handled in a faulty way.
```
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.evaluate.impl.ConfidenceBasedFMeasureCalculator] - <${pom_version} 0|||||||||>
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.evaluate.impl.ConfidenceBasedFMeasureCalculator] - <${pom_version} 1|||||||||>
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.evaluate.impl.ConfidenceBasedFMeasureCalculator] - <${pom_version} 2|||||||||>
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,610 [Thread-7] DEBUG [org.aksw.gerbil.matching.impl.MatchingsCounterImpl] -
2023-06-05 11:57:52,611 [Thread-7] INFO [org.aksw.gerbil.execute.ExperimentTask] -
```
The evaluation brings up the following two mismatches in the second and third document, respectively:
- `((78, 12, [http://dbpedia.org/resource/Bill_Clinton, http://www.ontologydesignpatterns.org/data/oke-challenge/task-1/Bill_Clinton])) != ((78, 12, [http://nl.dbpedia.org/resource/Bill_Clinton, http://nl.wikipedia.org/wiki/Bill_Clinton]))`
- `((44, 6, [http://www.ontologydesignpatterns.org/data/oke-challenge/task-1/Sydney, http://dbpedia.org/resource/Sydney])) != ((44, 6, [http://de.dbpedia.org/resource/Sydney, http://de.wikipedia.org/wiki/Sydney]))`
This raises several questions:
1. Why does the sameAs retrieval provide two different sets of IRIs although it gets the same input documents for the dataset and annotator?
2. Why do we only have language-dependent DBpedia IRIs instead of a larger set of IRIs?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running the D2KBTest and tracing the SameAs retrieval for the Bill Clinton and Sydney mismatches. Inspect the logged MatchingsCounterImpl results and the input documents from the OKE Task 1 Example Dataset. Done means explaining why the IRI sets differ and restoring the expected test behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100