About using label smoothing

Hello,

while training my retrieval model I used categorical cross entropy with label smoothing as my loss function in order not to punish high scores on (user, item) pairs that might be false negatives. When trying to combine it with the candidate_ids to remove accidental hits I got very high loss values.

By looking at your code I realized that this is due to the fact that RemoveAccidentalHits sets these "hits" to the lowest possible value (as log(x) tends to minus infinity as x tends to 0). However, when using label smoothing the way to remove these accidental hits is not by setting these values to be close to minus infinity but to the logarithm of label_smoothing/n_samples

I made a quick fix by myself but maybe you should consider this question.

The issue names RemoveAccidentalHits but no file or test. Start by tracing that component and how candidate_ids interact with categorical cross entropy and label smoothing. Confirm the expected accidental-hit behavior and add the corresponding regression coverage; the issue does not provide a concrete acceptance criterion.