NVIDIA-Merlin / NVIDIA-Merlin/Transformers4Rec

[FEA] Support feeding pre-trained embeddings to TF4Rec model with high-level api

Open
#475 8 comments 0 reactions 1 assignee View on GitHub

@rnyak is already working on this.

Since Jan 30, 2023.

P0 status/needs-triage
Dominant language
Python
Stars
1.3k
Forks
165
Avg merge
1m
Merged PRs (30d)
2

Description

🚀 Feature request

Currently we do not have out of the box support for adding pre-trained embeddings to embedding layer, and ability to freeze them, and train a TF4Rec model. We have embedding_initializer but we never tested if it works accurately and as expected. May be we can create in PyTorch a class like TensorInitializer (TF) as we did in Merlin Models and expose the embedding initializer and trainable args to the user.

We need to

  • Expose definition of embeddings module in the input blocks: TabularFeatures and TabularSequenceFeatures
  • Support feeding pre-trained embeddings to TF4Rec model with high-level api (users should be add them to the embedding layer, and freeze them, i.e., set trainable=False (TF Api) or requires_grad=False (PyTorch API))
  • create an example notebook for showcasing that functionality

Motivation

This is a FEA coming from our customers and users.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.