TREC iKAT 2023/2024

**Dataset Information:**

The purpose is to add the processed TREC iKAT collection (same collection for years 2023 and 2024 [subset of ClueWeb22-B]).
The Shared Task of iKAT can be defined as personalized retrieval-based "candidate response retrieval" in context of the conversation.
Collection with around 116,838,987 passages (with id in the form: clueweb22-en0004-50-00170:0).

**Links to Resources:**

Guidelines from year 2023: https://www.trecikat.com/guidelines/
Overview of year 2023: https://arxiv.org/abs/2401.01330
Github of year 2023: https://github.com/irlabamsterdam/iKAT
Test topics and qrels 2023: https://trec.nist.gov/data/ikat2023.html

**Dataset ID(s) & supported entities:**

We can provide with the documents, and flatten version of the conversation:
`trec_ikat23/doc` : collection of passages, 116M passages
`trec_ikat23/queries` : the flatten conversations (156 entrees from the 24 topics)
`trec_ikat23/qrels` : qrels from the flatten conversations

**Checklist**

Mark each task once completed. All should be checked prior to merging a new dataset.

- [x] Dataset definition (in `ir_datasets/datasets/[topid].py`)
- [x] Tests (in `tests/integration/[topid].py`)
- [x] Metadata generated (using `ir_datasets generate_metadata` command, should appear in `ir_datasets/etc/metadata.json`)
- [x] Documentation (in `ir_datasets/etc/[topid].yaml`)
- [ ] Documentation generated in https://github.com/seanmacavaney/ir-datasets.com/
- [x] Downloadable content (in `ir_datasets/etc/downloads.json`)
- [ ] Download verification action (in `.github/workflows/verify_downloads.yml`). Only one needed per `topid`.
- [ ] Any small public files from NIST (or other potentially troublesome files) mirrored in https://github.com/seanmacavaney/irds-mirror/. Mirrored status properly reflected in `downloads.json`.

**Additional comments/concerns/ideas/etc.**

The collection requires a licence approved by CMU, is it possible to restrict the access of the collection? (more details below)

> 💥 Document Collection: TREC iKAT 2023 ClueWeb22-B
>
> The collection distribution is being handled directly by CMU and not the iKAT organizers. Please follow these steps to get your data license ASAP:
>
> Sign the license form available on the [ClueWeb22 project web page](https://lemurproject.org/clueweb22/obtain.php).
> Send the form to CMU for approval (jlm4@andrew.cmu.edu)
>
> Please give enough time to the CMU licensing office to accept your request. A download link will be sent to you by the ClueWeb22 team at CMU.
>
> Note:
>
> CMU requires a signature from the organization (i.e., the university or company), not an individual who wants to use the data. This can slow down the process at your end too. So, it’s useful to start the process ASAP.
> If you already have an accepted license for ClueWeb22, you don’t need a new form. Please let us know if that’s the case.