alibaba / alibaba/x-deeplearning

单机模式下,一机多卡GPU显存占用问题

Open
#125 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
PureBasic
Stars
4.3k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

在xdl.python.backend.tf.tf_bachend.py中定义了
def tf_wrapper(is_training=True, init_grad=None, gpu_memory_fraction=0.5, device_type='cpu'):

小白有两个问题:
1、我单机上有4块GPU,用local模式运行可不可以自动全部占满显存?
2、目前我提交的任务只能占一块卡的0.5,改gpu_memory_fraction参数到0.8也没有变化(两个任务把第一块用完),为啥其他三块卡没用完呢?

![image](https://user-images.githubusercontent.com/16347334/52160407-ef93f080-26ef-11e9-93ad-4ecabe8b5efe.png)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading xdl.python.backend.tf.tf_bachend.py and the tf_wrapper function, then reproduce local-mode execution with four GPUs while varying gpu_memory_fraction. Determine whether the observed single-GPU allocation is expected or faulty; document the supported behavior or define a scoped fix once the GPU placement and memory configuration are understood.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.