lablup / lablup/backend.ai

GPU memory utilization pct of liveStat seems not correct

Open
#1,059 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

### What Operating System(s) are you seeing this problem on?

Linux (x86-64)

### Backend.AI version

22.09.11

### Describe the bug

I used the [live_stat.cuda_mem.pct](https://github.com/lablup/backend.ai/blob/main/src/ai/backend/manager/idle.py#L602) value to get the session's gpu memory live stat.But it seems not correct. So I just got the percent by divide `live_stat.cuda_mem.currrent` with `live_stat.cuda_mem.capacity`. However, this is a temporary measure and it would be better to provide the correct pct value from the server.Related links: https://github.com/lablup/giftbox/issues/128, https://github.com/lablup/backend.ai-webui/pull/1588

### To Reproduce

call [get_current_utilization](https://github.com/lablup/backend.ai/blob/main/src/ai/backend/manager/idle.py#L582).

### Expected Behavior

`live_stat.cuda_mem.capacity` value needs to be correct.

### Anything else?

_No response_

JIRA Issue: BA-221

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.