google-research / google-research/precondition
advice on how to reduce compile times?
Open
- Dominant language
- Jupyter Notebook
- Stars
- 34
- Forks
- 6
- Avg merge
- 7h 52m
- Merged PRs (30d)
- 2
Description
right now I have to wrap shampoo in `optax.flatten` or the tracing time on init is crazy long (284s in one case) and I don't know the compile time because I didn't wait long enough, but it's at least 10 minutes.
but wrapping in `optax.flatten` will reshard the entire parameter array every step, which is often going to be quite slow.
Contributor guide
Assessment
This issue has not been assessed yet.