huggingface / huggingface/accelerate
Be able to use `notebook_launcher` with TPUs on cloud TPU instances
Open
enhancement
TPU
wip
- Dominant language
- Python
- Stars
- 9.9k
- Forks
- 1.5k
- Avg merge
- 5d 2h
- Merged PRs (30d)
- 27
Description
Currently, the `notebook_launcher`[support TPUs only on Kaggle or Google Colab](https://github.com/huggingface/accelerate/blob/4e2c5117d7931544e53b11453fabd4ca427e6307/src/accelerate/launchers.py#LL60C5-L60C7). I would like to be able to use `notebook_launcher` on a cloud TPU instance.
We could introduce a the `distributed_type` as a parameter of the `notebook_launcher` and refactor `in_colab` and `in_kaggle` to `use_tpu` with addition if `distributed_type` is set.
Contributor guide
Assessment
This issue has not been assessed yet.