tensorflow / tensorflow/recommenders
Incremental retrieval model training with Hashing method
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2k
- Forks
- 300
- PR merge metrics
- No merged PRs in 30d
Description
I have developed a retrieval model for personalized movie recommendations. However, in the real world, new users and new content continue to emerge. To address this challenge, I have learned about the benefits of using hashing embedding.
In tutorial, I found the hashing layer was putted as part of the model architecture. why this can avoid retraining the model every time?
Besides, I don't know how to handle hashing collisions and determine the appropriate value for the num_bins parameter. In the provided example, even with only 5 inputs and setting num_bins to 6, 2 values (['b'],['c'] ) were still hashed to the same bin.
layer = layer = tf.keras.layers.Hashing(num_bins=6)
inp = [['a'], ['b'], ['c'], ['d'], ['e']]
layer(inp)
<tf.Tensor: shape=(5, 1), dtype=int64, numpy=
array([[3],
[4],
[4],
[5],
[1]])>
In my real codes, for example, I have 10000 user_id before, and each day will have around 1000 new users, how should I set the num_bins to ensure each user has their unique hashed code?
How about calculating the total number of users each day and setting the num_bins parameter to the number of users for that specific day? Will the old users still have the same hashed codes as before?
Thanks in advance.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the tutorial's hashing-layer section and run the provided tf.keras.layers.Hashing example. Trace how the example addresses new users, new content, collisions, num_bins, and hash stability, then document clear guidance for incremental retrieval-model training and the expected behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100