tensorflow / tensorflow/recommenders

Incremental retrieval model training with Hashing method

Open
#703 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2k
Forks
300
PR merge metrics
No merged PRs in 30d

Description

I have developed a retrieval model for personalized movie recommendations. However, in the real world, new users and new content continue to emerge. To address this challenge, I have learned about the benefits of using hashing embedding.

In tutorial, I found the hashing layer was putted as part of the model architecture. why this can avoid retraining the model every time?
Besides, I don't know how to handle hashing collisions and determine the appropriate value for the num_bins parameter. In the provided example, even with only 5 inputs and setting num_bins to 6, 2 values (['b'],['c'] ) were still hashed to the same bin.

layer = layer = tf.keras.layers.Hashing(num_bins=6)
inp = [['a'], ['b'], ['c'], ['d'], ['e']]
layer(inp)

<tf.Tensor: shape=(5, 1), dtype=int64, numpy=
array([[3],
       [4],
       [4],
       [5],
       [1]])>

In my real codes, for example, I have 10000 user_id before, and each day will have around 1000 new users, how should I set the num_bins to ensure each user has their unique hashed code?
How about calculating the total number of users each day and setting the num_bins parameter to the number of users for that specific day? Will the old users still have the same hashed codes as before?

Thanks in advance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the tutorial's hashing-layer section and run the provided tf.keras.layers.Hashing example. Trace how the example addresses new users, new content, collisions, num_bins, and hash stability, then document clear guidance for incremental retrieval-model training and the expected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.