dmlc / dmlc/decord

Importing `torch` after importing `decord` causes hanging behavior

Open
#263 6 comments 5 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.5k
Forks
232
PR merge metrics
No merged PRs in 30d

Description

I ran into a strange issue where the import order specifically between `torch` and `decord` can causing hanging behavior.

Concretely, I found that having your imports like this (which could happen by chance or if you use `isort`):
``` python
import decord
import torch # hangs :(
```
will cause a hang. However, switching the order of imports fixes the issue:
``` python
import torch
import decord
```

This seems like some issue with the way decord's imports are set up internally, though I'm not sure exactly what would cause this. This is what the trace looks like when it's truly hanging:
```
>(963): (1003): (1006): --- modulename: _bootstrap, funcname: _load_unlocked
(666): (668): (674): --- modulename: _bootstrap, funcname: module_from_spec
(567): (568): (571): --- modulename: _bootstrap_external, funcname: create_module
(1176): (1177): (1176): --- modulename: _bootstrap, funcname: _call_with_frames_removed
```

Steps to reproduce:
1. Create conda environment
```
conda create -p NEW_ENV_PATH python=3.10
conda activate NEW_ENV_PATH
pip install torch decord
```
2. Create interactive python session
3. Import decord: `import decord`
4. Attempt to import torch: `import torch`

I found this to be reproducible with the following settings:
```
>>> torch.__version__
'2.0.0+cu117'
>>> decord.__version__
'0.6.0'
```
OS: CentOS Linux Version 7 (Core)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the issue in the stated Python 3.10 conda environment with torch 2.0.0+cu117 and decord 0.6.0, trying both import orders. Trace the import path involved in the hang; done means importing decord followed by torch completes without hanging while the reverse order remains functional.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
developer-experience, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.