google-research / google-research/task_adaptation
How to run VTAB on multi-GPUs?
Open
- Dominant language
- Python
- Stars
- 352
- Forks
- 37
- PR merge metrics
- No merged PRs in 30d
Description
Thanks for your solid work!
I have some problem with reproducing the result on your leaderboard. The suggest batch size on your paper is 512 however the biggest batch size supported on one GPU(2080 ti) is 64 (128 will cause OOM).
The tensorflow_hub seems not working well with `tf.distribute.MirroredStrategy()` to support multi-GPUs, leading to the error below:
> RuntimeError: variable_scope module/ was unused but the corresponding name_scope was already taken.
just as mentioned in https://github.com/tensorflow/hub/issues/64
Would you please give me some suggestions? (Not familiar with tf very well)
Contributor guide
Assessment
This issue has not been assessed yet.