google-research / google-research/precondition

advice on how to reduce compile times?

Open
#24 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
34
Forks
6
Avg merge
7h 52m
Merged PRs (30d)
2

Description

right now I have to wrap shampoo in `optax.flatten` or the tracing time on init is crazy long (284s in one case) and I don't know the compile time because I didn't wait long enough, but it's at least 10 minutes.

but wrapping in `optax.flatten` will reshard the entire parameter array every step, which is often going to be quite slow.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.