facebookresearch / facebookresearch/dlrm

Unable to preprocess Criteo Kaggle Display Advertising Challenge Dataset

Open
#374 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.1k
Forks
859
PR merge metrics
No merged PRs in 30d

Description

I have downloaded and unzipped the 4GB dataset. It consists of 3 files, readme.txt, train.txt, and test.txt. It is stored in a folder called dataset, which is a sibling folder to my cloned dlrm folder.

I believe this is the command needed to preprocess, as implied in the README

python dlrm_s_pytorch.py --raw-data-file=../dataset/train.txt

however, the output of this is

world size: 1, current rank: 0, local rank: 0
Using CPU...
time/loss/accuracy (if enabled):
Finished training it 1/1 of epoch 0, -1.00 ms/it, loss 0.083850

I have seen another issue, #274 where someone posted lines that should happen when preprocessing occurs.
Reading raw data=/my_raw_data_path/train.txt

Additionally, there is no .npz file(s) in my input directory.

Is it because there are some other required flags?

Contributor guide

Open the contributing guide

Research direction

Start with the README's preprocessing instructions and the entry point in dlrm_s_pytorch.py, then compare the documented command with the reported training output. Check how train.txt and test.txt are expected to be supplied and where preprocessing should produce .npz files. Done means the documented preprocessing path runs against the downloaded dataset and produces the expected files.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.