dice-group / dice-group/gerbil
[QA] Implement unit test for experiment with uploaded files for QA system and dataset
- Dominant language
- Java
- Stars
- 231
- Forks
- 55
- PR merge metrics
- No merged PRs in 30d
Description
# Problem
The situation in which a system answer file and a dataset file are uploaded occurs often in practice but there is no unit test covering it. Hence, bugs like the one fixed with commit https://github.com/dice-group/gerbil/commit/794c41a14d73ba554ae28a397d415db0d68ae67b are hard to detect.
# Solution
Implement a unit test that simulates an experiment in which 2 files have been uploaded and are used as prediction and gold standard.
The implementation should be done on the following branch, which still has the bug mentioned above: [QA-name-bug](https://github.com/dice-group/gerbil/tree/QA-name-bug)
The unit test should simulate an experiment task that is created by submitting the following configuration from the UI:
```json
{"type":"QA","matching":"STRONG_ENTITY_MATCH","annotator":[],"dataset":["NIFDS_gold1(gold_1.json)"],"answerFiles":["AF_System(pred_1.json)(undefined)(AFDS_gold_1.json)"],"questionLanguage":"en"}
```
`pred_1.json` and `gold_1.json` should have the same content and may have to be moved into the upload directory as preparation of the unit test. It is expected that the resulting F1-measures are 1.0.
[GoldStdEvalTest](https://github.com/dice-group/gerbil/blob/QA-name-bug/src/test/java/org/aksw/gerbil/execute/GoldStdEvalTest.java) can be used as inspiration.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.