Add BioASQ dataset to the list of supported BEIR datasets
- Lenguaje dominante
- Python
- Estrellas
- 391
- Forks
- 58
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
Hi @seanmacavaney I would like to use the BioASQ dataset for an experiment and I have stumbled across this on the GitHub repo of the BEIR paper [beir-cellar](https://github.com/beir-cellar/beir/issues/86#issuecomment-1548959460) where the author links the preprocessed data for the 4 datasets marked as "unavailable". I am aware that you've been trying to extend the list of available datasets from the benchmark on ir_datasets (ie. this [issue](https://github.com/allenai/ir_datasets/issues/58)) and I was wondering if, given these resources, BioASQ could be integrated to the catalog?
**Dataset Information:**
BioASQ is a dataset featuring in the [BEIR](https://openreview.net/forum?id=wCu6T5xFjeJ) benchmark and originated from a challenge around "biomedical semantic indexing and question answering". More information about the challenge and the dataset can be found here: http://bioasq.org/
**Links to Resources:**
Link to the steps listed on [beir-cellar](https://github.com/beir-cellar/beir) in order to reproduce the files: https://github.com/beir-cellar/beir/tree/main/examples/dataset#2-bioasq ;
Link to the Google Drive space linked in the issue cited above where the preprocessed data can be found: https://drive.google.com/drive/folders/1CgDO-KmQQMpGEGeD3R20ZgTTM008xix9
**Dataset ID(s) & supported entities:**
- `beir/bioasq-2020`: queries, docs
- `beir/bioasq-2020/train`: queries, docs, qrels
- `beir/bioasq-2020/test`: queries, docs, qrels
**Checklist**
Mark each task once completed. All should be checked prior to merging a new dataset.
- [ ] Dataset definition (in `ir_datasets/datasets/[topid].py`)
- [ ] Tests (in `tests/integration/[topid].py`)
- [ ] Metadata generated (using `ir_datasets generate_metadata` command, should appear in `ir_datasets/etc/metadata.json`)
- [ ] Documentation (in `ir_datasets/etc/[topid].yaml`)
- [ ] Documentation generated in https://github.com/seanmacavaney/ir-datasets.com/
- [ ] Downloadable content (in `ir_datasets/etc/downloads.json`)
- [ ] Download verification action (in `.github/workflows/verify_downloads.yml`). Only one needed per `topid`.
- [ ] Any small public files from NIST (or other potentially troublesome files) mirrored in https://github.com/seanmacavaney/irds-mirror/. Mirrored status properly reflected in `downloads.json`.
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.