google-research / google-research/big_vision
Error with putting arrays on CPU in cloud TPUs
- Dominant language
- Jupyter Notebook
- Stars
- 3.5k
- Forks
- 227
- PR merge metrics
- No merged PRs in 30d
Description
Hi I've been setting up big vision on a v4-32 TPU pod and I run into this error whenever I call `u.put_cpu`
jaxlib.xla_extension.XlaRuntimeError: INVALID_ARGUMENT: Cannot copy array to non-addressable device TFRT_CPU_0
I'm guessing the CPUs on the TPU pod aren't configured properly? Is there a way around this or a way to fix this issue?
Totally new to TPUs and let me know if you need more information.
Contributor guide
Research direction
Start by reproducing the `u.put_cpu` call on the v4-32 TPU pod and capture the full `XlaRuntimeError` context. Compare the behavior with the TPU and CPU device configuration; the issue is done when the cause and a verified workaround or fix are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100