facebookresearch / facebookresearch/dlrm
Unable to preprocess Criteo Kaggle Display Advertising Challenge Dataset
- Dominant language
- Python
- Stars
- 4.1k
- Forks
- 859
- PR merge metrics
- No merged PRs in 30d
Description
I have downloaded and unzipped the 4GB dataset. It consists of 3 files, readme.txt, train.txt, and test.txt. It is stored in a folder called dataset, which is a sibling folder to my cloned dlrm folder.
I believe this is the command needed to preprocess, as implied in the README
python dlrm_s_pytorch.py --raw-data-file=../dataset/train.txt
however, the output of this is
world size: 1, current rank: 0, local rank: 0
Using CPU...
time/loss/accuracy (if enabled):
Finished training it 1/1 of epoch 0, -1.00 ms/it, loss 0.083850
I have seen another issue, #274 where someone posted lines that should happen when preprocessing occurs.
Reading raw data=/my_raw_data_path/train.txt
Additionally, there is no .npz file(s) in my input directory.
Is it because there are some other required flags?
Contributor guide
Research direction
Start with the README's preprocessing instructions and the entry point in dlrm_s_pytorch.py, then compare the documented command with the reported training output. Check how train.txt and test.txt are expected to be supplied and where preprocessing should produce .npz files. Done means the documented preprocessing path runs against the downloaded dataset and produces the expected files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100