[Question] Candidate sampling probability for Mixed Negative Sampling

Hi Team,

Sorry for posting many questions, I hope this will help others navigate through building high quality models :)

I've trained a two tower model, and checking the possibility of using it directly, but from inspecting a few outputs I guess part of the retrieval is focused on popular items even if it's not that relevant to the user context (previous searches, previous items, previous purchases, time of the day .. etc), I'm not sure if i need to add more features, tune it further .. etc (will do so anyway).

But I guess using Mixed Negative Sampling would help with that, as its main target is to make the model better understand how to distinguish between frequent items and non-frequent items --> so it'll be a bit more confident in ranking non-popular items higher when it see the context to, but I'm wondering how to calculate the candidate sampling probability, for the two tower model, i calculated it using the item frequency in training set (# appearences/ # all interactions)

Quoting from the paper, it seems that the way we should be calculating the candidate_sampling_probability should be different as the We're sampling the other batch uniformly, but how it should be different?
My guess that it would be a weighted average of

sampling_probability = (batch_1 * item_frequency_in_training + batch_2 * uniform_frequency) / (batch_1 + batch_2)
batch_1: the original batch size for two tower model
batch_2: the complementary random negative batch size

Can you please confirm my understanding for MNS impact (retrieving more high quality results)? and the way to calculate the sampling_probability?

Thanks :)

Start with the linked paper and the issue’s description of mixed negative sampling for a two-tower model. No source files or tests are mentioned; done would be a clear explanation of the method’s impact and how to calculate candidate_sampling_probability when combining the original and uniformly sampled batches.