[Bug]: RunInference GPU error: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!
Open
bug
core
ml
P2
python
run-inference
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What happened?
When using multiple processes (when we don’t use `no_use_multiple_sdk_containers`) in DataFlow jobs, this error has been observed
`Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!`
The current solution is to add the `experiment=no_use_multiple_sdk_containers` flag.
We should look into how much benefit we get with multiple processes, as opposed to maxing out GPU utilization in one worker.
### Issue Priority
Priority: 2
### Issue Component
Component: sdk-py-core
Contributor guide
Assessment
This issue has not been assessed yet.