allenai / allenai/ir_datasets

LongEval Retrieval (used at CLEF 2023)

Đang mở
#234 6 bình luận 0 reaction 0 người được giao Xem trên GitHub
add-dataset
Ngôn ngữ chính
Python
Star
391
Fork
58
Chỉ số merge pull request
Không có pull request nào được merge trong 30 ngày

Mô tả

**Dataset Information:**

The goal would be to integrate the data of LongEval for the task 1 on retrieval.

The information from the [official task description](https://clef-longeval.github.io/tasks/):

```
The goal of Task 1 is to propose an information retrieval system which can handle changes over the time. The proposed retrieval system should follow the temporal timewise evolution of Web documents. The Longeval Websearch collection relies on a large set of data (corpus of pages, queries, user interaction) provided by a commercial search engine (Qwant). It is designed to reflect the changes of the Web across time, by providing evolving document and query sets. The queries in the collection were collected from Qwant's users over several months and can thus be expected to reflect the changes in the search preferences of the users. The documents in the collection were then selected to be able to well evaluate retrieval on these queries at the time they were collected, and thus also change over a time.
```

**Links to Resources:**

https://clef-longeval.github.io/

**Dataset ID(s) & supported entities:**

- `longeval/en/train`: docs, queries, qrels
- `longeval/en/heldout`: docs, queries
- `longeval/en/a-short-july`: docs, queries
- `longeval/en/b-long-september`: docs, queries
- `longeval/fr/train`: docs, queries, qrels
- `longeval/fr/heldout`: docs, queries
- `longeval/fr/a-short-july`: docs, queries
- `longeval/fr/b-long-september`: docs, queries

**Checklist**

Mark each task once completed. All should be checked prior to merging a new dataset.

- [ ] Dataset definition (in `ir_datasets/datasets/[topid].py`)
- [ ] Tests (in `tests/integration/[topid].py`)
- [ ] Metadata generated (using `ir_datasets generate_metadata` command, should appear in `ir_datasets/etc/metadata.json`)
- [ ] Documentation (in `ir_datasets/etc/[topid].yaml`)
- [ ] Documentation generated in https://github.com/seanmacavaney/ir-datasets.com/
- [ ] Downloadable content (in `ir_datasets/etc/downloads.json`)
- [ ] Download verification action (in `.github/workflows/verify_downloads.yml`). Only one needed per `topid`.
- [ ] Any small public files from NIST (or other potentially troublesome files) mirrored in https://github.com/seanmacavaney/irds-mirror/. Mirrored status properly reflected in `downloads.json`.

**Additional comments/concerns/ideas/etc.**

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.