Update PyTorch to Flax porting documentation to include information on weight initialisation
Open
- Dominant language
- Jupyter Notebook
- Stars
- 7.3k
- Forks
- 833
- Avg merge
- 5h 11m
- Merged PRs (30d)
- 5
Description
Stumbled across google/jax#4862 discussion on parameter initialisation differences between PyTorch and Flax.
Would it be worth adding a note in the documentation that highlights these weight init differences? Users porting training scripts may expect the initialisation to be the same but find training behaviour is different as a result of different initialisation.
The issue discusses Linear and Conv layers mainly, but there are differences in other layers such as embedding layers.
Would be willing to contribute this myself if you can point me to / provide a quick guide on contributing to docs. Thanks :)
Contributor guide
Assessment
This issue has not been assessed yet.