huggingface / huggingface/optimum-graphcore
Allow setting max-weight-norm optimizer parameter
Open
enhancement
- Dominant language
- Python
- Stars
- 87
- Forks
- 33
- PR merge metrics
- No merged PRs in 30d
Description
[From AlexC in GC]
The reference implementation uses a value of 10 for this parameter. However the implementation in GC-Optimum passes `None`, with no possibility for the user to set this parameter
- create training command line option `--max-weight-norm` it can default to `None` for backwards compatibility
- in `IPUTrainer.create_optimizer` pass this argument to LAMB optimizer as part of `optimizer_kwargs`
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.