dmlc / dmlc/decord

Deadlock while running on GPU

Open
#187 1 comment 6 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.5k
Forks
232
PR merge metrics
No merged PRs in 30d

Description

Hi all,
first, thanks for this great work.

I encounter a weird issue while testing decord on GPU on my computer.
I am running the following benchmark command:
> `python3 bench_decord.py --file /my_video.mp4` for CPU
> `python3 bench_decord.py --file /my_video.mp4 --gpu 0` for GPU

It is working fine on CPU ending with the following line:
> `300 frames, elapsed time for random access(accurate): 25.084531545639038`

But on GPU it deadlocks almost right away after this line:
> `[h264 @ 0x171d080] mmco: unref short failure`

This appends while running localy or through docker (from decord/gpu.Dockerfile) with nvdia-runtime enabled

Here is my nvidia-smi output (its the same locally or from inside the docker):
> `NVIDIA-SMI 470.63.01 Driver Version: 470.63.01 CUDA Version: 11.4`

From inside the docker I don't see any process lines in nvdia-smi, but locally I crealy see the process line jumping onto the GPU as here:
> `| 0 N/A N/A 5444 C python3 205MiB |`

GPU usage keep very low, and it deadlocks for ever.

Do I have to switch to another version of CUDA maybe ?

Thanks a lot.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.