bigscience-workshop / bigscience-workshop/xmtf
Questions on creating instruction data
- Dominant language
- Jupyter Notebook
- Stars
- 534
- Forks
- 43
- PR merge metrics
- No merged PRs in 30d
Description
Thanks for the great work!
I have a few questions regarding data creation of xP3 after following the guide [here](https://github.com/bigscience-workshop/xmtf#create-xp3) to create instruction data on the `code` language subset.
1. I noticed the total samples of the public processed data (from [here](https://huggingface.co/datasets/bigscience/xP3all)) on the `code` split is **2707724**. However, my resulting data following the above github guide is much more than that (approximately >3M samples). I wonder if there were any additional post-processing to get the final instruction data for tuning?
2. Following the above github guide, I noticed there was no prompt for this particular dataset State Changes. I got this warning when running the [creation code](https://github.com/bigscience-workshop/bigscience/blob/master/data/xp3/prepare_xp3_train.py):
``Tried instantiating `DatasetTemplates` for Fraser/python-state-changes, but no prompts found. Please ignore this warning if you are creating new prompts for this dataset.``
Is this dataset not assigned with any prompt (similar to how HumanEval was treated). Or is the below version of PromptSource I used is not correct:
``git clone -b tr13 https://github.com/Muennighoff/promptsource.git & install cd promptsource; pip install -e .``
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.