facebookresearch / facebookresearch/dlrm

[Question] Best ACC & AUC Never Get Updated in Inference State / Function?

Open
#216 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.1k
Forks
859
PR merge metrics
No merged PRs in 30d

Description

https://github.com/facebookresearch/dlrm/blob/9c2fda79afbc09e277c17e420bffe510125b4f70/dlrm_s_pytorch.py#L1353-L1357
https://github.com/facebookresearch/dlrm/blob/9c2fda79afbc09e277c17e420bffe510125b4f70/dlrm_s_pytorch.py#L750-L759
https://github.com/facebookresearch/dlrm/blob/9c2fda79afbc09e277c17e420bffe510125b4f70/dlrm_s_pytorch.py#L880-L890

From the above code snippets, it looks like to me that the values **best_acc_test** and **best_auc_test** never get updated as it only stays in the inference function scope.

Is it designed for that? or if it is actually a bug?

Contributor guide

Open the contributing guide

Research direction

Read dlrm_s_pytorch.py at lines 750-759, 880-890, and 1353-1357, then trace how best_acc_test and best_auc_test are created and used during inference. Done means determining whether their scope is intentional and documenting the conclusion or identifying the precise behavior that needs correction.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.