训练一段时间后报错
- Dominant language
- Python
- Stars
- 36.9k
- Forks
- 5.2k
- PR merge metrics
- No merged PRs in 30d
Description
经常训练到1200步左右的时候就报错:
Traceback (most recent call last):
File "synthesizer_train.py", line 37, in
train(**vars(args))
File "E:\MockingBird-main\synthesizer\train.py", line 180, in train
for i, (texts, mels, embeds, idx) in enumerate(data_loader, 1):
File "E:\python3.7\lib\site-packages\torch\utils\data\dataloader.py", line 352, in __iter__
return self._get_iterator()
File "E:\python3.7\lib\site-packages\torch\utils\data\dataloader.py", line 294, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "E:\python3.7\lib\site-packages\torch\utils\data\dataloader.py", line 801, in __init__
w.start()
File "E:\python3.7\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "E:\python3.7\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "E:\python3.7\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "E:\python3.7\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
reduction.dump(process_obj, to_child)
File "E:\python3.7\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
File "E:\python3.7\lib\multiprocessing\synchronize.py", line 104, in __getstate__
h = context.get_spawning_popen().duplicate_for_child(sl.handle)
File "E:\python3.7\lib\multiprocessing\popen_spawn_win32.py", line 95, in duplicate_for_child
return reduction.duplicate(handle, self.sentinel)
File "E:\python3.7\lib\multiprocessing\reduction.py", line 77, in duplicate
0, inheritable, _winapi.DUPLICATE_SAME_ACCESS)
PermissionError: [WinError 5] 拒绝访问。
PS E:\MockingBird-main> Traceback (most recent call last):
File "", line 1, in
File "E:\python3.7\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "E:\python3.7\lib\multiprocessing\spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at synthesizer_train.py line 37 and synthesizer/train.py line 180, then inspect the PyTorch DataLoader iteration and its Windows multiprocessing path. Reproduce training to around step 1200 on the reported Python 3.7 Windows setup, identify why worker startup raises WinError 5 and EOFError, and confirm that training continues without the failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100