tensorflow / tensorflow/recommenders
[Question] two-tower-model + infoNCE how to optimize
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2k
- Forks
- 300
- PR merge metrics
- No merged PRs in 30d
Description
I have tried a two-tower model (user and query) in a real industrial scenario using contrastive learning. The samples are all actual click samples, and the loss function is InfoNCE. I have a few questions:
- The model performs best with only one layer, and the more MLP layers I add, the worse the HR@100 becomes.
- Using L2 normalization at the end of the model degrades performance.
As a result, I currently only have one MLP layer and no normalization. Could you please provide some advice or share some experiences on what I should do?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file, test, or entry point is named. Begin by locating the relevant two-tower and InfoNCE implementation, then define completion as an evidence-based response addressing the reported effects of MLP depth and L2 normalization.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100