google / google/flax

Update PyTorch to Flax porting documentation to include information on weight initialisation

Open
#3,125 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
7.3k
Forks
833
Avg merge
5h 11m
Merged PRs (30d)
5

Description

Stumbled across google/jax#4862 discussion on parameter initialisation differences between PyTorch and Flax.

Would it be worth adding a note in the documentation that highlights these weight init differences? Users porting training scripts may expect the initialisation to be the same but find training behaviour is different as a result of different initialisation.

The issue discusses Linear and Conv layers mainly, but there are differences in other layers such as embedding layers.

Would be willing to contribute this myself if you can point me to / provide a quick guide on contributing to docs. Thanks :)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.