shenweichen / shenweichen/DeepCTR-Torch
希望model.fit中train和val的metric可以单独设置
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.5k
- Forks
- 733
- PR merge metrics
- No merged PRs in 30d
Description
problem：
在二分类任务下，model.fit过程中，使用validation_data，使用auc作为metric，设置verbose>0，如果train的正负样本极度不均衡，在部分batch中会出现所有样本的标签一致，从而导致在train_data上auc计算报错（basemodel.py 248-249行）。但是如果不使用auc作为metric，则观察不到validation_data在训练过程中的auc变化。
solution:
希望将train和val使用的metric可以分别单独设置，实现仅在validation_data上计算metric
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading model.fit and basemodel.py lines 248-249, focusing on how metrics are configured and computed for training and validation_data. Determine how separate train and validation metrics should be represented, then verify that validation AUC can be reported without calculating it on batches whose labels are all the same.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100