Save BERT model separately
- Dominant language
- Python
- Stars
- 452
- Forks
- 76
- PR merge metrics
- No merged PRs in 30d
Description
Currently, we save all model parameters together. This leads to a problem that if the code is changed, we will not be able to load the model. What's even worse is that the failure happens silently.
This problem becomes more serious when we need to fine-tune the BERT model. It is common that we need to fine-tune the BERT model and load the fine-tuned BERT model to train GNN model. It is expensive to fine-tune the BERT model. Once it is tuned, we prefer to keep using it even if the GraphStorm code is changed.
To solve this problem, we should save the BERT model separately from the remaining model parameters.
Contributor guide
Research direction
No files, tests, or entry points are named in the issue. Start by locating the model serialization and BERT fine-tuning load paths, then trace how the remaining parameters are saved. Done means BERT is stored separately, can be reused after GraphStorm code changes, and loading failures are no longer silent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100