awslabs / awslabs/graphstorm

Avoid storing the input embeddings whenever necessary

Open
#492 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
452
Forks
76
PR merge metrics
No merged PRs in 30d

Description

Currently, if the input encoder is a BERT encoder, `require_cache_embed` is True. This triggers `compute_node_input_embeddings` and tries to store all embeddings from the input encoder. In a heterogeneous graph, some of the nodes may require to compute BERT embeddings while other nodes don't need. We should only cache the embeddings of the nodes with text features.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the require_cache_embed path into compute_node_input_embeddings, focusing on how BERT input embeddings are handled for heterogeneous graph nodes. Done means embeddings are cached only for nodes with text features, while nodes without text features are not stored.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.