Megvii-BaseDetection / Megvii-BaseDetection/YOLOX

subprocess.CalledProcessError: Command '['python3', '/home/b/hxb/yolox/YOLOX/tools/train.py']' returned non-zero exit status 1.

Open
#320 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
10.6k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

有人能帮我解决一下问题吗?
我用pycharm链接服务器后,在pycharm上运行train.py会遇到以下问题:
ssh://b@222.197.000.10:9022/home/b/miniconda3/envs/yolo1.7/bin/python -u /home/b/hxb/yolox/YOLOX/tools/train.py
2021-08-03 09:34:38.959 | INFO | yolox.core.launch:launch_by_subprocess:145 -


Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.


Traceback (most recent call last):
File "/home/b/hxb/yolox/YOLOX/tools/train.py", line 5, in
from loguru import logger
ModuleNotFoundError: No module named 'loguru'
Traceback (most recent call last):
File "/home/b/hxb/yolox/YOLOX/tools/train.py", line 5, in
from loguru import logger
ModuleNotFoundError: No module named 'loguru'
Traceback (most recent call last):
File "/home/b/hxb/yolox/YOLOX/tools/train.py", line 135, in
args=(exp, args),
File "/home/b/hxb/yolox/YOLOX/yolox/core/launch.py", line 87, in launch
args,
File "/home/b/hxb/yolox/YOLOX/yolox/core/launch.py", line 165, in launch_by_subprocess
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['python3', '/home/b/hxb/yolox/YOLOX/tools/train.py']' returned non-zero exit status 1.
我尝试了一下,如果在train.py中修改devices参数来只使用单卡的话,便不会报这个错误。但当我使用两张卡时,这个错误便出现了。
如果将from loguru import logger 放在 import torch后面,则会报错:
ssh://b@222.197.219.10:9022/home/b/miniconda3/envs/yolo1.7/bin/python -u /home/b/hxb/yolox/YOLOX/tools/train.py
2021-08-03 09:46:28.464 | INFO | yolox.core.launch:launch_by_subprocess:145 -


Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.


Traceback (most recent call last):
File "/home/b/hxb/yolox/YOLOX/tools/train.py", line 7, in
import torch
ModuleNotFoundError: No module named 'torch'
Traceback (most recent call last):
File "/home/b/hxb/yolox/YOLOX/tools/train.py", line 7, in
import torch
ModuleNotFoundError: No module named 'torch'
Traceback (most recent call last):
File "/home/b/hxb/yolox/YOLOX/tools/train.py", line 135, in
args=(exp, args),
File "/home/b/hxb/yolox/YOLOX/yolox/core/launch.py", line 87, in launch
args,
File "/home/b/hxb/yolox/YOLOX/yolox/core/launch.py", line 165, in launch_by_subprocess
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['python3', '/home/b/hxb/yolox/YOLOX/tools/train.py']' returned non-zero exit status 1.
运行的代码是昨天下刚载的。
如果直接用Xshell在服务器上运行train.py则没有这个问题。

我的环境:
CUDA10.1 虚拟环境:python 3.7 pytorch==1.7.0 torchvision==0.8.0 torchaudio==0.7.0 cudatoolkit=10.1

希望有人能帮我解答一下,谢谢!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tools/train.py and yolox/core/launch.py, especially launch_by_subprocess, to trace how the multi-GPU workers are started. Compare the interpreter and environment used by the subprocess with the PyCharm and Xshell environments, checking imports for loguru and torch. Done means multi-GPU training starts with the intended environment without ModuleNotFoundError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.