tensorflow / tensorflow/models
Multi GPU not being utilized
Open
@pkulzc is already working on this.
Since Jun 22, 2020.
models:research:odapi
type:support
- Dominant language
- Python
- Stars
- 77.7k
- Forks
- 44.8k
- PR merge metrics
- No merged PRs in 30d
Description
System information
- What is the top-level directory of the model you are using: object-detection
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): nvidia-docker with nvidia/cuda 10 and cuDNN 7
- TensorFlow installed from (source or binary): binary
- TensorFlow version (use command below): 1.14.0
- Bazel version (if compiling from source): N/A
- CUDA/cuDNN version: 7
- GPU model and memory: GTX 1080 8GB
- Exact command to reproduce: ?
Describe the problem
When training with multiple GPUs, GPU 0 is being utilized while utilization of all other GPUs sit at 0%. I believe the desired behavior is that more GPUs would offer some performance advantage over a single GPU
Source code / logs
models/research/object-detection/legacy/train.py
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.