possible max_grad_norm implementation error
Open
Beginner friendly
- Dominant language
- Python
- Stars
- 452
- Forks
- 76
- PR merge metrics
- No merged PRs in 30d
Description
Clipping is currently applied after the optimizer step. The gradient clipping code controlled by the `max-grad-norm` cli param should be implemented between the backward and step calls.
# Node Prediction Code
https://github.com/awslabs/graphstorm/blob/f3a063669aba254889d4902f6f5c7d512401632e/python/graphstorm/trainer/np_trainer.py#L214-L221
Contributor guide
Research direction
Start in python/graphstorm/trainer/np_trainer.py around lines 214-221 and trace the node prediction training flow from backward through optimizer step. Confirm that max-grad-norm clipping occurs between those calls, then verify the node prediction path still trains correctly with the CLI option enabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 62/100