DropoutNet - Use official config & sample data but AUC and loss worsen with more training steps
- Dominant language
- Python
- Stars
- 2.4k
- Forks
- 383
- PR merge metrics
- No merged PRs in 30d
Description
## Description
I attempted to train the official DropoutNet model using the provided sample Taobao dataset and the sample configuration file. However, during training, I observed that the AUC decreased and the losses increased as the training steps progressed. Based on my understanding, the expected behavior is that the AUC should increase and the losses should decrease as training continues.
## Steps to reproduce
OS: Ubuntu 20.04
GPU: 1 NVIDIA RTX 3090
Python: 3.10.16
TensorFlow: 2.14.0 with CUDA
1. git clone the easyrec repo (commit SHA: https://github.com/alibaba/EasyRec/commit/4b0b1f5a2a990b253737ec532611fa4f9387d372)
2. install easyrec
3. download the sample taobao dataset:
```shell
wget http://easyrec.oss-cn-beijing.aliyuncs.com/data/git_oss_sample_data/data_test_tb_data_b1579db090d72b3b70b59ba3c7692701 -O tb_data.tar.gz
tar -zxf tb_data.tar.gz
```
4. run the training with the [sample dropoutnet config](https://github.com/alibaba/EasyRec/blob/4b0b1f5a2a990b253737ec532611fa4f9387d372/samples/model_config/dropoutnet_on_taobao.config) and sample dataset
```shell
python -m easy_rec.python.train_eval --pipeline_config_path samples/model_config/dropoutnet_on_taobao.config
```
## Actual training result
TensorBoard:
```shell
tensorboard --logdir experiments/dropoutnet_taobao_ckpt/eval_val
```

Initial AUC and loss:

Final AUC and loss:

## Expected behavior
- AUC should increase with more training steps.
- Losses should decrease with more training steps.
Could you please confirm if this is expected behavior or if there might be an issue with the sample configuration or dataset? If additional debugging information is needed, I am happy to provide more details.
Thank you!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running easy_rec.python.train_eval with samples/model_config/dropoutnet_on_taobao.config and the downloaded sample Taobao dataset. Inspect the TensorBoard logs under experiments/dropoutnet_taobao_ckpt/eval_val and compare AUC and loss across training steps. Done means determining whether the result is expected or providing evidence that the sample configuration or dataset is faulty.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100