DAMO-NLP-SG / DAMO-NLP-SG/LLM-R2
Lack of Evaluation Datasets
- Dominant language
- Python
- Stars
- 54
- Forks
- 13
- PR merge metrics
- No merged PRs in 30d
Description
I encountered the same issue and found a related problem worth noting.
The error occurs because the evaluation dataset files (`sts-dev.csv`, `sts-test.csv`)
under `data/data_simcse/{dataset}/` are not included in the repository.
`prepare_CL_dataset.py` is responsible for generating these files, but it also requires
`training_triplets_{dataset}_total.csv` as input — which is likewise absent from the repo.
To reproduce the full training pipeline for datasets other than TPC-H (e.g., DSB, JOB),
could you provide:
1. The `training_triplets_*_total.csv` files (or the script used to generate them from the pool CSVs), and
2. The pre-trained SimCSE models for DSB and JOB, as used in the paper's experiments?
I would greatly appreciate your help in fully reproducing the results.
Thank you.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.