babysor / babysor/MockingBird

训练一段时间后报错

Open
#449 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
36.9k
Forks
5.2k
PR merge metrics
No merged PRs in 30d

Description

经常训练到1200步左右的时候就报错:

Traceback (most recent call last):
File "synthesizer_train.py", line 37, in
train(**vars(args))
File "E:\MockingBird-main\synthesizer\train.py", line 180, in train
for i, (texts, mels, embeds, idx) in enumerate(data_loader, 1):
File "E:\python3.7\lib\site-packages\torch\utils\data\dataloader.py", line 352, in __iter__
return self._get_iterator()
File "E:\python3.7\lib\site-packages\torch\utils\data\dataloader.py", line 294, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "E:\python3.7\lib\site-packages\torch\utils\data\dataloader.py", line 801, in __init__
w.start()
File "E:\python3.7\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "E:\python3.7\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "E:\python3.7\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "E:\python3.7\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
reduction.dump(process_obj, to_child)
File "E:\python3.7\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
File "E:\python3.7\lib\multiprocessing\synchronize.py", line 104, in __getstate__
h = context.get_spawning_popen().duplicate_for_child(sl.handle)
File "E:\python3.7\lib\multiprocessing\popen_spawn_win32.py", line 95, in duplicate_for_child
return reduction.duplicate(handle, self.sentinel)
File "E:\python3.7\lib\multiprocessing\reduction.py", line 77, in duplicate
0, inheritable, _winapi.DUPLICATE_SAME_ACCESS)
PermissionError: [WinError 5] 拒绝访问。
PS E:\MockingBird-main> Traceback (most recent call last):
File "", line 1, in
File "E:\python3.7\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "E:\python3.7\lib\multiprocessing\spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at synthesizer_train.py line 37 and synthesizer/train.py line 180, then inspect the PyTorch DataLoader iteration and its Windows multiprocessing path. Reproduce training to around step 1200 on the reported Python 3.7 Windows setup, identify why worker startup raises WinError 5 and EOFError, and confirm that training continues without the failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.