Importer crash in parallel import on windows.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 2.1k
- Avg merge
- 4d 21h
- Merged PRs (30d)
- 31
Description
I think the threaded implementation is doing something unsafe where it is moving a temp file and then grabbing an incorrect one.
```sh
Traceback (most recent call last):
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\util\__init__.py", line 508, in move
os.replace(syspath(path), syspath(dest))
FileNotFoundError: [WinError 2] The system cannot find the file specified: '\\\\?\\E:\\Incoming\\Mindscape – Black Lotus (2007) {CITRUSCD002} [FLAC]\\CD 01 (Unmixed)\\01 Mindscape & Jade & Hydro - Black Lotus (Break Remix).flac' -> '\\\\?\\E:\\Media Library\\Music\\Digital\\m\\Mindscape\\Black Lotus (2007, Citrus Recordings CITRUSCD002) [FLAC 16-44]\\1-01 - Mindscape + Jade + Hydro - Black Lotus (Break Remix).1.flac'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "c:\users\henry\.local\bin\beet.exe\__main__.py", line 5, in
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\ui\__init__.py", line 1645, in main
_raw_main(args)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\ui\__init__.py", line 1624, in _raw_main
subcommand.func(lib, suboptions, subargs)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\ui\commands\import_\__init__.py", line 131, in import_func
import_files(lib, byte_paths, query)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\ui\commands\import_\__init__.py", line 75, in import_files
session.run()
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\importer\session.py", line 237, in run
pl.run_parallel(QUEUE_SIZE)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\util\pipeline.py", line 471, in run_parallel
raise exc_info[1].with_traceback(exc_info[2])
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\util\pipeline.py", line 383, in run
self.coro.send(msg)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\util\pipeline.py", line 195, in coro
task = func(*args, task)
^^^^^^^^^^^^^^^^^
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\importer\stages.py", line 299, in manipulate_files
task.manipulate_files(
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\importer\tasks.py", line 483, in manipulate_files
item.move(operation)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\library\models.py", line 1164, in move
self.move_file(dest, operation)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\library\models.py", line 1049, in move_file
util.move(self.path, dest)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\util\__init__.py", line 520, in move
with open(syspath(path), "rb") as f:
^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '\\\\?\\E:\\Incoming\\Mindscape – Black Lotus (2007) {CITRUSCD002} [FLAC]\\CD 01 (Unmixed)\\01 Mindscape & Jade & Hydro - Black Lotus (Break Remix).flac'
```
I've gotten this issue several times importing files with beets installed at commit https://github.com/beetbox/beets/commit/6dbee2437e39eba386735a757e58a8309f8c9c69
I'll try finishing the rest of this import session with non-threaded and see what happens.
Contributor guide
Research direction
Start by reading beets/util/pipeline.py and importer/session.py around run_parallel, then trace the file handling through importer/tasks.py, library/models.py, and util/__init__.py. Reproduce a Windows parallel import and compare it with a non-threaded import; done means the importer no longer loses or selects the wrong source file.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100