allenai / allenai/ir_datasets

TREC NeuCLIR 2022

Abierto
#164 4 comentarios 1 reacción 0 asignados Ver en GitHub
add-dataset
Lenguaje dominante
Python
Estrellas
391
Forks
58
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

**Dataset Information:**

"The main task for the proposed track is ad-hoc cross-language retrieval. Documents will be drawn from Common Crawl newswire, and will be written in Chinese, Russian, and Persian. Topics will be in English, and will be expressed in traditional TREC title/description/narrative form. Retrieved documents will be graded as highly relevant, somewhat relevant, and not relevant; we expect to use several metrics to evaluate runs, including nDCG@100 and ERR."

**Links to Resources:**

- https://neuclir.github.io/
- Related: HC4

**Dataset ID(s) & supported entities:**

- TBD

**Checklist**

Mark each task once completed. All should be checked prior to merging a new dataset.

- [ ] Dataset definition (in `ir_datasets/datasets/[topid].py`)
- [ ] Tests (in `tests/integration/[topid].py`)
- [ ] Metadata generated (using `ir_datasets generate_metadata` command, should appear in `ir_datasets/etc/metadata.json`)
- [ ] Documentation (in `ir_datasets/etc/[topid].yaml`)
- [ ] Documentation generated in https://github.com/seanmacavaney/ir-datasets.com/
- [ ] Downloadable content (in `ir_datasets/etc/downloads.json`)
- [ ] Download verification action (in `.github/workflows/verify_downloads.yml`). Only one needed per `topid`.
- [ ] Any small public files from NIST (or other potentially troublesome files) mirrored in https://github.com/seanmacavaney/irds-mirror/. Mirrored status properly reflected in `downloads.json`.

**Additional comments/concerns/ideas/etc.**

cc: @eugene-yang

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.