tensorflow / tensorflow/recommenders
[Question] How best to re-use large, repeated text features.
@maciejkula is already working on this.
Since Dec 14, 2020.
- Dominant language
- Python
- Stars
- 2k
- Forks
- 300
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
From reading the tutorials, the way data is passed into the model would mean lots of repetition, for example in the movie example, you would keep passing the same movie many times for different users. If we have a movie summary, this would mean having to store that summary many times over - which can get pretty big.
user | movie | movie summary
My question is what is the best way to create a "FeatureLookup" of sorts where for the same movie name / id we could grab the summary which would be stored once, and pass this in during training / inference?
tf.keras.Sequential([
tf.keras.layers.experimental.preprocessing.StringLookup(
vocabulary=unique_movies, mask_token=None
),
tf.keras.layers.FeatureLookup(), # Fake layer, here as example, returns movie summary.
])
Thanks in advance!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.