google / google/flax

HOWTOS tracking issue

Open
#2,116 3 comments 0 reactions 1 assignee Claimed by @marcvanzee View on GitHub
Priority: P2 - no schedule Status: pull requests welcome
Dominant language
Jupyter Notebook
Stars
7.3k
Forks
833
Avg merge
5h 11m
Merged PRs (30d)
5

Description

This issue tracks which HOWTOs we would like to add.

## Process
- If you like to work on HOWTO, please create a PR for it and mention this issue in the PR description, once it is merged we will check the box below.
- If you think we should add a HOWTO, please reply to this issue and we will add it to the list below.

## HOWTOs

- [ ] Data-parallel training. @gmittal mentioned they would like to add this based on #1982.
- [ ] Best practices for dynamic length inputs.
- [ ] Loading MNIST from torchvision and HuggingFace dataset (see #1853 for more details).
- [ ] Correctly dealing with the last batch during eval (see #1850 for more details).
- [ ] Gradient checkpointing.
- [ ] Using `nn.apply` and `nn.bind` (See #1087).
- [ ] Mixed precision training (suggested by @lkhphuc).
- [ ] Dropout guide (similar to BatchNorm guide)
- [ ] How to load from different datasets: torch, tf.data, HuggingFace and explain that in Flax we really only care about jax numpy arrays.
- [ ] How to do gradient accumulation
- [ ] Freezing parameters
- [ ] Training with multiple optimizers
- [ ] Gradient checkpointing.
- [ ] Flax RNG Design
- [ ] Using scan-over-layers to trade off peak memory with speed
- [ ] How to use Module.bind()

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.