Megvii-BaseDetection / Megvii-BaseDetection/YOLOX

TypeError: __init__() missing 1 required positional argument: 'dtype'

Open
#416 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
10.6k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

My training command: python tools/train.py -f exps/example/yolox_voc/yolox_voc_s.py -d 1 -b 1 --fp16 -o -c weights/yolox_s.pth.tar
My training log:
2021-08-08 20:00:58.315 | INFO | yolox.core.trainer:before_epoch:193 - ---> start train epoch1
2021-08-08 20:00:58.315 | INFO | yolox.core.trainer:after_train:188 - Training of experiment is done and the best AP is 0.00
2021-08-08 20:00:58.316 | ERROR | yolox.core.launch:launch:90 - An error has been caught in function 'launch', process 'MainProcess' (9236), thread 'MainThread' (9960):
Traceback (most recent call last):

File "tools/train.py", line 125, in
args=(exp, args),
│ └ Namespace(batch_size=1, ckpt='weights/yolox_s.pth.tar', devices=1, dist_backend='nccl', dist_url=None, exp_file='exps/example...
└ ╒══════════════════╤═════════════════════════════════════════════════════════════════════════════════════════════════════════...

File "d:\mmdetection\yolox\yolox-main\yolox\core\launch.py", line 90, in launch
main_func(*args)
│ └ (╒══════════════════╤════════════════════════════════════════════════════════════════════════════════════════════════════════...
└ <function main at 0x000001E02979ED38>

File "tools/train.py", line 104, in main
trainer.train()
│ └ <function Trainer.train at 0x000001E03F0B9318>
└ <yolox.core.trainer.Trainer object at 0x000001E04024AF88>

File "d:\mmdetection\yolox\yolox-main\yolox\core\trainer.py", line 71, in train
self.train_in_epoch()
│ └ <function Trainer.train_in_epoch at 0x000001E03F3C90D8>
└ <yolox.core.trainer.Trainer object at 0x000001E04024AF88>

File "d:\mmdetection\yolox\yolox-main\yolox\core\trainer.py", line 80, in train_in_epoch
self.train_in_iter()
│ └ <function Trainer.train_in_iter at 0x000001E03F3C3288>
└ <yolox.core.trainer.Trainer object at 0x000001E04024AF88>

File "d:\mmdetection\yolox\yolox-main\yolox\core\trainer.py", line 86, in train_in_iter
self.train_one_iter()
│ └ <function Trainer.train_one_iter at 0x000001E0402123A8>
└ <yolox.core.trainer.Trainer object at 0x000001E04024AF88>

File "d:\mmdetection\yolox\yolox-main\yolox\core\trainer.py", line 92, in train_one_iter
inps, targets = self.prefetcher.next()
│ │ └ <function DataPrefetcher.next at 0x000001E03F3C3048>
│ └ <yolox.data.data_prefetcher.DataPrefetcher object at 0x000001E0497091C8>
└ <yolox.core.trainer.Trainer object at 0x000001E04024AF88>

File "d:\mmdetection\yolox\yolox-main\yolox\data\data_prefetcher.py", line 48, in next
self.preload()
│ └ <function DataPrefetcher.preload at 0x000001E03F3C1F78>
└ <yolox.data.data_prefetcher.DataPrefetcher object at 0x000001E0497091C8>

File "d:\mmdetection\yolox\yolox-main\yolox\data\data_prefetcher.py", line 30, in preload
self.next_input, self.next_target, _, _ = next(self.loader)
│ │ │ │ │ └ <torch.utils.data.dataloader._MultiProcessingDataLoaderIter object at 0x000001E0495AAE88>
│ │ │ │ └ <yolox.data.data_prefetcher.DataPrefetcher object at 0x000001E0497091C8>
│ │ │ └ tensor([[[ 11.0000, 529.3131, 509.4199, 221.3737, 261.1603],
│ │ │ [ 14.0000, 369.5336, 320.0000, 540.9327, 640.0000],
│ │ │ ...
│ │ └ <yolox.data.data_prefetcher.DataPrefetcher object at 0x000001E0497091C8>
│ └ tensor([[[[ 0.0878, 0.2258, 0.1085, ..., -0.3814, -0.3814, -0.3814],
│ [ 0.0257, 0.2189, 0.1568, ..., -0.3814, ...
└ <yolox.data.data_prefetcher.DataPrefetcher object at 0x000001E0497091C8>

File "D:\Anaconda\envs\yolox2\lib\site-packages\torch\utils\data\dataloader.py", line 435, in next
data = self._next_data()
│ └ <function _MultiProcessingDataLoaderIter._next_data at 0x000001E03EB90318>
└ <torch.utils.data.dataloader._MultiProcessingDataLoaderIter object at 0x000001E0495AAE88>

File "D:\Anaconda\envs\yolox2\lib\site-packages\torch\utils\data\dataloader.py", line 1085, in _next_data
return self._process_data(data)
│ │ └ <torch._utils.ExceptionWrapper object at 0x000001E04A1D66C8>
│ └ <function _MultiProcessingDataLoaderIter._process_data at 0x000001E03EB90438>
└ <torch.utils.data.dataloader._MultiProcessingDataLoaderIter object at 0x000001E0495AAE88>

File "D:\Anaconda\envs\yolox2\lib\site-packages\torch\utils\data\dataloader.py", line 1111, in _process_data
data.reraise()
│ └ <function ExceptionWrapper.reraise at 0x000001E02B690318>
└ <torch._utils.ExceptionWrapper object at 0x000001E04A1D66C8>

File "D:\Anaconda\envs\yolox2\lib\site-packages\torch_utils.py", line 428, in reraise
raise self.exc_type(msg)
│ │ └ 'Caught MemoryError in DataLoader worker process 1.\nOriginal Traceback (most recent call last):\n File "D:\Anaconda\envs...
│ └ <class 'numpy.core._exceptions._ArrayMemoryError'>
└ <torch._utils.ExceptionWrapper object at 0x000001E04A1D66C8>

TypeError: init() missing 1 required positional argument: 'dtype'
I don't know how to solve it

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the training command in tools/train.py and trace the failing data-loading path through yolox/core/trainer.py and yolox/data/data_prefetcher.py. Reproduce the failure, inspect the nested NumPy ArrayMemoryError in the DataLoader worker traceback, and consider the issue resolved when the command can load a batch and continue training without the reported exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.