NVIDIA-Merlin / NVIDIA-Merlin/Transformers4Rec
[FEA] Support feeding pre-trained embeddings to TF4Rec model with high-level api
Open
@rnyak is already working on this.
Since Jan 30, 2023.
P0
status/needs-triage
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 165
- Avg merge
- 1m
- Merged PRs (30d)
- 2
Description
🚀 Feature request
Currently we do not have out of the box support for adding pre-trained embeddings to embedding layer, and ability to freeze them, and train a TF4Rec model. We have embedding_initializer but we never tested if it works accurately and as expected. May be we can create in PyTorch a class like TensorInitializer (TF) as we did in Merlin Models and expose the embedding initializer and trainable args to the user.
We need to
- Expose definition of embeddings module in the input blocks: TabularFeatures and TabularSequenceFeatures
- Support feeding pre-trained embeddings to TF4Rec model with high-level api (users should be add them to the embedding layer, and freeze them, i.e., set trainable=False (TF Api) or requires_grad=False (PyTorch API))
- create an example notebook for showcasing that functionality
Motivation
This is a FEA coming from our customers and users.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.