Avoid storing the input embeddings whenever necessary
Open
- Dominant language
- Python
- Stars
- 452
- Forks
- 76
- PR merge metrics
- No merged PRs in 30d
Description
Currently, if the input encoder is a BERT encoder, `require_cache_embed` is True. This triggers `compute_node_input_embeddings` and tries to store all embeddings from the input encoder. In a heterogeneous graph, some of the nodes may require to compute BERT embeddings while other nodes don't need. We should only cache the embeddings of the nodes with text features.
Contributor guide
Research direction
Start by tracing the require_cache_embed path into compute_node_input_embeddings, focusing on how BERT input embeddings are handled for heterogeneous graph nodes. Done means embeddings are cached only for nodes with text features, while nodes without text features are not stored.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100