beetbox / beetbox/beets

Importer crash in parallel import on windows.

Open
#6,360 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
15.7k
Forks
2.1k
Avg merge
4d 21h
Merged PRs (30d)
31

Description

I think the threaded implementation is doing something unsafe where it is moving a temp file and then grabbing an incorrect one.
```sh
Traceback (most recent call last):
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\util\__init__.py", line 508, in move
os.replace(syspath(path), syspath(dest))
FileNotFoundError: [WinError 2] The system cannot find the file specified: '\\\\?\\E:\\Incoming\\Mindscape – Black Lotus (2007) {CITRUSCD002} [FLAC]\\CD 01 (Unmixed)\\01 Mindscape & Jade & Hydro - Black Lotus (Break Remix).flac' -> '\\\\?\\E:\\Media Library\\Music\\Digital\\m\\Mindscape\\Black Lotus (2007, Citrus Recordings CITRUSCD002) [FLAC 16-44]\\1-01 - Mindscape + Jade + Hydro - Black Lotus (Break Remix).1.flac'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "c:\users\henry\.local\bin\beet.exe\__main__.py", line 5, in
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\ui\__init__.py", line 1645, in main
_raw_main(args)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\ui\__init__.py", line 1624, in _raw_main
subcommand.func(lib, suboptions, subargs)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\ui\commands\import_\__init__.py", line 131, in import_func
import_files(lib, byte_paths, query)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\ui\commands\import_\__init__.py", line 75, in import_files
session.run()
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\importer\session.py", line 237, in run
pl.run_parallel(QUEUE_SIZE)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\util\pipeline.py", line 471, in run_parallel
raise exc_info[1].with_traceback(exc_info[2])
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\util\pipeline.py", line 383, in run
self.coro.send(msg)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\util\pipeline.py", line 195, in coro
task = func(*args, task)
^^^^^^^^^^^^^^^^^
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\importer\stages.py", line 299, in manipulate_files
task.manipulate_files(
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\importer\tasks.py", line 483, in manipulate_files
item.move(operation)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\library\models.py", line 1164, in move
self.move_file(dest, operation)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\library\models.py", line 1049, in move_file
util.move(self.path, dest)
File "C:\Users\Henry\AppData\Local\pipx\pipx\venvs\beets\Lib\site-packages\beets\util\__init__.py", line 520, in move
with open(syspath(path), "rb") as f:
^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '\\\\?\\E:\\Incoming\\Mindscape – Black Lotus (2007) {CITRUSCD002} [FLAC]\\CD 01 (Unmixed)\\01 Mindscape & Jade & Hydro - Black Lotus (Break Remix).flac'
```

I've gotten this issue several times importing files with beets installed at commit https://github.com/beetbox/beets/commit/6dbee2437e39eba386735a757e58a8309f8c9c69

I'll try finishing the rest of this import session with non-threaded and see what happens.

Contributor guide

Open the contributing guide

Research direction

Start by reading beets/util/pipeline.py and importer/session.py around run_parallel, then trace the file handling through importer/tasks.py, library/models.py, and util/__init__.py. Reproduce a Windows parallel import and compare it with a non-threaded import; done means the importer no longer loses or selects the wrong source file.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.